So I changed my algorithm so that HMM can accept different size of sequence.
(currently, I normalize the size and then calculates log-likelihood)
After that, I could get reasonable result.
You can download graph here.
Sunday, April 25, 2010
Analyzied result of mine
After I draw some graphs, I found that my result is not consistent with Andrew's.
Presentation outline (maybe for report/paper too?)
#1: Hook (X stroke rate, Y death rate. Obviously rehab is important)
#2: Need (Current method relies on time only. Can we do better [qualitative]?)
#2a: show video of select exercises
#3: Approach
#3a: Use classification algorithms, but choose such that they give similarity scores
#3b: Obvious choices: DTW (cost) and HMM (log likelyhood). Others: ANN, decision tree: no score just classify. Naive Bayes: might work (probability)
#3c: This is basically gesture recognition
#3c-1: Problems: usually related to orientation and speed/amplitude
#3d: But we're solving a different problem because
#3d-1: We know the action a-priori, the amplitude is fixed (you have to perform the action as specified in the test) and we can still use time.
#3e: So this lets us work with the path data not just the acceleration
#3f: DTW on the path
#3g: HMM on the path - novel state space
#3g-1: and most importantly we can fix the orientation
#4: Implementation details
#4a: application
#4b: remote control
#4c: algorithms implemented on phone: DTW and HMM (log-likelyhood)
#4c-1: and 'energy' (current method used)
#5: Results
#5a: similar actions = similar scores
#5b: Show DTW and HMM graphs
#5c: Experiments with weights and 'poor motion'
#5d: W/ weights, not conclusion, with 'poor motion' pretty good results
#5d-1: Energy is inconclusive even for 'poor motion'
#5e: Ex. #5 added after other work to show the method wasn't tuned to ex. 1-4
#5f: 'High-box' tested to look at similar actions. Scores are similar, but we're not classifying, similar actions get similar scores.
#6: Conclusion Future work
#6a: We do a more quantitative measurement of the actions similarity
#6a-1: On an easy to use, inexpensive system (with remote control)
#6b: But our experiment is a bit un-scientific
#6c: So future directions are:
#6c-1: Can existing kinematic captures of patients be used to test our system
#6c-2: And of course if we could test it on a few actual patients.
#2: Need (Current method relies on time only. Can we do better [qualitative]?)
#2a: show video of select exercises
#3: Approach
#3a: Use classification algorithms, but choose such that they give similarity scores
#3b: Obvious choices: DTW (cost) and HMM (log likelyhood). Others: ANN, decision tree: no score just classify. Naive Bayes: might work (probability)
#3c: This is basically gesture recognition
#3c-1: Problems: usually related to orientation and speed/amplitude
#3d: But we're solving a different problem because
#3d-1: We know the action a-priori, the amplitude is fixed (you have to perform the action as specified in the test) and we can still use time.
#3e: So this lets us work with the path data not just the acceleration
#3f: DTW on the path
#3g: HMM on the path - novel state space
#3g-1: and most importantly we can fix the orientation
#4: Implementation details
#4a: application
#4b: remote control
#4c: algorithms implemented on phone: DTW and HMM (log-likelyhood)
#4c-1: and 'energy' (current method used)
#5: Results
#5a: similar actions = similar scores
#5b: Show DTW and HMM graphs
#5c: Experiments with weights and 'poor motion'
#5d: W/ weights, not conclusion, with 'poor motion' pretty good results
#5d-1: Energy is inconclusive even for 'poor motion'
#5e: Ex. #5 added after other work to show the method wasn't tuned to ex. 1-4
#5f: 'High-box' tested to look at similar actions. Scores are similar, but we're not classifying, similar actions get similar scores.
#6: Conclusion Future work
#6a: We do a more quantitative measurement of the actions similarity
#6a-1: On an easy to use, inexpensive system (with remote control)
#6b: But our experiment is a bit un-scientific
#6c: So future directions are:
#6c-1: Can existing kinematic captures of patients be used to test our system
#6c-2: And of course if we could test it on a few actual patients.
Graphs
I think I've got most of my graphs made. See here.
Also, the gist of the data collected is that purposely screwing up the motion to emulate a patient causes a large increase in the score. This is good. It doesn't always increase the energy. This is also good since energy is sort of the current measure used, we can say we're better.
Jin: can you make some 3D plots of the paths for some of our 'poor motion' examples?
Otherwise I think we're good.
Also, the gist of the data collected is that purposely screwing up the motion to emulate a patient causes a large increase in the score. This is good. It doesn't always increase the energy. This is also good since energy is sort of the current measure used, we can say we're better.
Jin: can you make some 3D plots of the paths for some of our 'poor motion' examples?
Otherwise I think we're good.
Friday, April 23, 2010
HMM result
| | Normal HMM (All) | Weight HMM (All) | Weight (Andrew) | Weight (Jin) | Weight (Scott) |
| Task1 | 81.5394 | 87.9466 | 101.2397 | 84.4787 | 81.2795 |
| Task2 | 82.3386 | 95.0149 | 95.3747 | 84.4096 | 105.0517 |
| Task3 | 79.2707 | 92.4030 | 109.0333 | 88.4455 | 83.9879 |
| Task4 | 87.8758 | 88.6020 | 92.6539 | 91.2206 | 84.0128 |
This is the result from HMM. (small number means good motion - close to trained motion)
As you can see, almost all weighted motions are higher than normal motions.
Scott's task4's score is better when he put on weight band. And it made normal HMM score and weight HMM score similar. Except for this, we can conclude that HMM seems reflect motion's quality well.
-Jin
Thursday, April 22, 2010
Energy results
The energy results are not very encouraging. Exercise #1 seems to be the most effected. Again, it is hard to know how relevant to a stroke patient these results are.
Normal:
Ex: Energy
--------------
1 168.53
2 66.442
3 94.084
4 483.74
Weighted:
Ex: Energy
--------------
1 221.3
2 57.46
3 56.263
4 205.54
DTW results from weighted test
While not a homerun, the results are interesting, see below. The take-away from all of this is that the weights probably changed our motion in a way the DTW algorithm could detect, but the result is not consistent.
So for exercises #1 and #4, the DTW score is lower for the unweighted. Unfortunately, for the others it is higher. However, for both of those exercises the major motion is lifting of the arm, so perhaps the weights matter more there. Also, we don't know how good of an analogue the weights are, so keep that in mind.
Also, it depends on the person:
Andrew
Jin
Scott
So for exercises #1 and #4, the DTW score is lower for the unweighted. Unfortunately, for the others it is higher. However, for both of those exercises the major motion is lifting of the arm, so perhaps the weights matter more there. Also, we don't know how good of an analogue the weights are, so keep that in mind.
Ex. Avg. Normal Avg. With Weights
------------------------------------
1 2079.35 4877.96
2 5925.08 3834.49
3 20313.27 17464.42
4 15911.03 18382.09
Also, it depends on the person:
Andrew
Ex. With Weights (min,mean,max,(stdev))
---------------------
1 1654.56, 2404.40, 3048.16 (438.37)
2 378.62, 957.23, 2641.28, (657.71)
3 17518.23, 26805.36, 41087.51 (5961.64)
4 7375.40, 17831.87, 32354.52 (8689.43)
Jin
Ex. With Weights
---------------------
1 1731.9, 3284.3, 4548.3 (771.31)
2 3947.8, 5788.8, 9190.8 (1820.9)
3 4298, 10568, 20295 (4663.9)
4 4806.2, 9506, 14470 (3355.6)
Scott
Ex. With Weights
---------------------
1 2464.1, 8945.2, 20658 (6588.5)
2 2658.5, 4757.4, 8741.8 (1936)
3 8992.3, 15020, 19028 (3275.4)
4 14906, 27808, 46056 (10216)
Subscribe to:
Posts (Atom)