HUMAN COMPUTER INTERACTIONAugmented Dressing using Microsoft Kinect Project TeamKhwaja Talha (13k-2253)Shaikh Zia (13k-2277)Abdul Moiz (12k-2144)Submitted to:Farrukh HassanNational University of Computer & Emerging Sciences (FAST-NU)ContentsINTRODUCTION 3DESIGN SURVEY 3NEW SYSTEM 3DESIGN DESCRIPTION 4SYSTEM ANALYSIS AND REQUIREMENTS GATHERING 5FUNCTIONAL REQUIREMENTS 5NON-FUNCTIONAL REQUIREMENTS 6DESIGN 6DETAILED GUI DESIGN 7INTERFACE DESIGN 10USABILITY CRITERIA 10HCI PRINCIPLES FOLLOWED 11USER EVALUATION 12EXPERT EVALUATION 13HEURISTIC EVALUATION 15INTRODUCTIONAugmented Dressing is a type of Windows application software that helps a customer for trying out a garment virtually on him/her, this project works on a Kinect device, the Kinect device works on IR and depth sensors and a camera, IR sensor detects user movement and joints, depth sensor detects the depth of the user and the camera helps in identifying the difference between the human skin and the virtual garment. Kinect device captures a dynamic video of user’s body on runtime which is mounted above the screen that helps the customer to track his/her body after which the user’s skeleton as well as the skin layer (through camera) is detected, where he/she would be standing right next to the screen. This skeletal tracking occurs with the help of Microsoft Kinect device attached with the adapter (9J7-00001) for Windows & a Camera will help to detect the skin colour of the user. After the detection process, the user will be seeing a 3D moveable image of the dress on screen overlapped on his/her body. The image of the garment that the user sees beside his runtime image is already saved in the database of the application and during the runtime of the application will be extracted from within the system.
So, when the user moves within a field of view of the screen, the image of the garment will also move accordingly. Hence an automated solution is proposed for the customers that shop regularly for garments. It measures the skeletal of the human body by the nodal points (also called joints) that are being pre-programmed on the Kinect device. For this program we use the spine centre as the centre of the human body for easy tracking. The customer can then try on virtual garments for the outlet store so that he can choose from variety of garments easily and without the fuss of anything.DESIGN SURVEYWhen a person wants to buy some clothes (like shirt/dress/pants etc.) He has to check the garment by following methods: a) He waits for his turn in the trial room in the garment store and then tries on his cloth pieces inside the room which takes longer time than usual as the person is changes the clothes and he has to see how the cloth pieces looks on them.
b) He buys the garments without trying. Which is a risk that the person may not like the cloth pieces afterwards as he has not tried them on himself because of any reason (maybe he is getting late from work /he is scared of changing clothes in public room because of a hidden camera/any other reason). NEW SYSTEMOur product offers a safe environment as in many countries misuse of trial room is done by placing a hidden/spy camera resulting in leaked videos or disrespect of a person especially women in Muslim countries, also customers face unnecessary fuss by changing his/her clothes and product theft issues occurs as well. This product also helps save time as the person does not have to change cloth pieces / he has to wait for his turn as the trial room is occupied. This project will try to eliminate all such issues as the customers of the specific brand will try the cloth pieces of outlet virtually. The customer easily tries on virtual cloth pieces of the garment outlet on them without the hassle of waiting for their turn especially on special occasions as most outlets close their trial rooms because of the abundance of customers present in the store and mainly because of hidden cameras in trial room.
It will also be space efficient as it requires just a screen and a Kinect device.DESIGN DESCRIPTIONFor augmented fitting, there would be two comparisons that take place:• in Depth Skeletal Tracking • Skin Detection With the help of Kinect Sensor, the skeletal tracker estimates the spatial coordinates and depth of body joints when the user comes closer to the screen while the sensor is attached on top of it, the Sensor will compare and extract the depth skeletal tracking just to find out physical presence of the user, the skin detection occurs with the help of camera in the Kinect device.All detection and tracking technologies operate using the following stages:Capture A physical or behavioural sample is captured by the system during the detection process. Extraction Skin and Skeleton tracking are extracted from the sample and a template is created. ComparisonThe template is then compared with a new sample (i.e. an augmented image of dress).
Image Overlapping After extracting the user’s data, the augmented image (dress) which will be selected by the customer will be overlapped on his/her body displayed on screen. And the customer can choose from the variety of dresses.SYSTEM ANALYSIS AND REQUIREMENTS GATHERINGBasically, the project team has used different requirements gathering techniques in order to collect the requirements of the system.
By the use of these techniques the project team has full idea that what to do. Interview/SurveyWe surveyed and interviewed different clothing brands (Khaadi and Outfitters etc.) to ensure that the different product requirements are fulfilled and the product runs efficiently. (Note: these surveys were done before the development of the project in order to gain information about the working of the project)Focus GroupWe also gathered together with different people to gain the product’s requirements and feedback, we collected feedback about, opportunities, needs and problems to refine and validate the already elicited requirements of the productFunctional RequirementsUser ExtractionThe user is extracted and isolated from the background to create an augmented reality environment.Skeletal TrackingThe skeletal tracker estimates the spatial coordinates and depth of body joints.Skin DetectionThe camera will help to detect the skin colour of a person make the layers of virtual dress.Image OverlappingThe augmented image (dress) which was selected by the customer will be overlapped on his/her body displayed on screen.Data AcquisitionThe customer selects the dress of his/her choice by virtually swiping his hand over the screen through the Kinect sensor.
Non-Functional RequirementsPerformance RequirementsThe Kinect sensor must work correctly and efficiently and images must be overlapped precisely, no lagging should occur, all the user extraction process should be precise also.Static RequirementsThese Requirements don’t impose any constraint on the execution characteristics of system. They are:No. of usersThe number of software may vary as this software finds application in all department of organization.Dynamic RequirementsThese specify constraints on the execution characteristics of the system, which include response time and throughput of the system. These factors are also applicable for the proposed software and it will suffice if the response time is high and the transactions are carried out precisely and quickly. The speed of the system is also fast it will quickly capture the face of student and quickly mark the attendance at the backend.Security RequirementsAs it is desktop based application software which runs in the shop only, it does not need any security or integrity, customer authorization is not needed as well because the software keeps running from opening till closing of the shop.
DESIGNThe overall design of this system will be broken into two different components: • Cloth Images • User interfaceThese components will work together to form the Augmented Dressing System. In the beginning, the system captures the user through the Kinect device and then the user will select the dress (cloth image) displayed on the right side of the interface by virtually swiping his/her hand over the screen through the Kinect sensor.DETAILED GUI DESIGN This image shows a side posture with the digital t shirt on the user. Above image is of a cloth piece (pants) on. The nodal joint (hip center) is tracked and afterwards the pants are placed on the person. Above image shows the actual T posture which helps in detecting the users body joints using Euclidean distance formula and the shirt is overlapped on the user.
Above image shows a red dress for female users. This GUI screen shows the movement of one arm while the other arm movement is static. This image shows the back of the user (as the program shows a 3D movement of the shirt that is overlapped on the user).
INTERFACE DESIGNA number of guidelines were followed for designing the interface: Use bigger fonts for clearer display of information. Use good contrast in the display of elements in the interface for clearer display of information. Every use action results in some change in the interface – color change of buttons when the mouse hovers on it, the note gets enlarged when the user tries to drag it.
Audio feedback for all the user actions – button clicks, transfer done, error in counting etc. All the garments used in the interface to give a real life feeling to the user.USABILITY CRITERIAThe developed prototype has fulfilled the initial usability criteria.• EffectivenessThe prototype has proved to be quite effective. All the user requirements and functional requirements are working very effectively.• EfficiencyThe system is efficient as it saves time and effort, if kept in comparison to the existing system.• SatisfactionMost of the users and subjects were satisfied with the performance of the new system. HCI PRINCIPLES FOLLOWED AffordanceAs its clear in the figure that the finished product will look somewhat like a mirror and it will therefore give an impression that one should come in front of it to see how he looks.
A novice user can perceive it very easily. So, the finished product to be made is developed keeping in view of the important principle of affordance.Reduced Cognitive OverheadThe GUI or wrapper application is simple and effective. There are no useless details or lesser important fuss on the screen real estate. There are virtually no long walkthroughs. The application and GUI are fairly simple. And this is made so in order to reduce Cognitive Overhead.Low Physical EffortThe system is almost effortless, you just come in front of the screen, stand as a ‘T’ and all set to go.
The previous system is far more effort consuming in comparison to this one. Perhaps the motive behind developing the Augmented system as to reduce time and effort.LearnabilityAs the GUI is developed quite simple keeping in view of the Cognitive overhead, learnability is also good. One just has to appear in front of the screen and select garments of choice with hand gestures just like he would have used a touchscreen.ResponsivenessThe only problem that the prototype is facing and receiving complaints against is the responsiveness. A few users have notices some lag in the system.
As this is just a prototype, this problem will be dealt with in the fully developed system.Error ToleranceThere are virtually no chances of errors in the usability of the system. Although the design of the garment would have some fitting issues. This will be fixed in the fully developed system.USER EVALUATIONType of group: In Group.No of users: Evaluated on 2 groups of 2 non-expert users each.The experiment is designed so in order to eliminate learning bias.
Hypothesis1) the Augmented system is easy to use and2) is time efficient.Quantitative Analysis The first group changed a similar shirt in the normal change room while the other group used our system to see how the shirt looks on them.Independent variable: Time Dependent variable: System usedConstants: Same type of garmentObservations of first partUser Change Room (time) User Augmented Reality (time)1. 3 min 10 sec 1. 28 secs2.
3 min 13 sec 2. 31 secsaverage Approx. 3 min 10 sec Approx.
30 secsThe second-time group B changed a similar shirt in the normal change room while the other group used our system to see how the shirt looks on them.Independent variable: time Dependent variable: system usedConstants: same type of garmentObservations of second part:User Change Room (time) User Augmented Reality (time)1. 3 min 01 sec 1. 28 secs2. 3 min 4 sec 2. 26 secsaverage Approx. 3 min Approx.
27 secsConclusionIn both of the experiments (with learning or without), the time taken to change a garment using augmented reality system is noticeably less. Perhaps it takes 1/6th of the time a user needs in an ordinary changing room.QUALITATIVE ANALYSISThese users were also asked to evaluate the ease of use, learnability and satisfaction. Most of them were very positive about the system.
Most user marked the system as effective, efficient and satisfactory.A few also pointed out some deficiencies in the system, that it lags and that the variety of designed garments is not sufficient. But both of these problems will be tackled as the prototype develops into the real deployable system.EXPERT EVALUATIONType of group: In Group.
No of users: Evaluated on 1 expert (Augmented Reality Developers) user.Hypothesis1) the Augmented system is easy to use and2) is time efficient.3) the 3D poses of garment fits nicely if cloth’s model is designed by expert designer.Quantitative AnalysisThe user changed a similar shirt in the normal change room to see how the shirt looks on them.Independent variable: time Dependent variable: system usedConstants: same type of garmentObservations of first partUser Change Room (time)1. 3 min 10 sec2. 3 min 13 secaverage Approx. 3 min 10 secThe second-time user used our system to see how the same shirt looks on them that he wore in the changing room.
Independent variable: time Dependent variable: system usedConstants: same type of garmentObservations of second partUser Augmented Reality (time)1. 28 secs2. 26 secs Approx. 27 secsConclusionIn both of the experiments (with learning or without), the time taken to change a garment using augmented reality system is noticeably less. Perhaps it takes 1/6th of the time a user needs in an ordinary changing room. Though the expert user complained about the performance issues as once the cloth overlaps on user’s body, it lags due to hardware limitations. QUALITATIVE ANALYSISThis expert user is also asked to evaluate the ease of use, learnability and satisfaction just as in non-expert users. He was very positive about the system and with such idea and solution to the replacement of changing room.
He marked the system as effective, efficient and satisfactory.Again, he pointed out the same that was mentioned by the non-expert users related to some deficiencies in the system, that it lags and that the variety of designed garments is not sufficient. But both of these problems will be tackled as the prototype develops into the real deployable system.HEURISTIC EVALUATIONS. No.
Process Heuristic Description Severity1 Calibration Text on Top of Screen in Black Consistency User will have to make a T-pose to calibrate their body joints 02 Clothing Selection Menu on Right Side of Screen Consistency User selects cloth by gripping the hand cursor virtually 13 Model Overlapped on User’s Body Error Prevention User can see virtually that how the garment looks on him/her 3