Presentation outline (maybe for report/paper too?)

#1: Hook (X stroke rate, Y death rate. Obviously rehab is important)

#2: Need (Current method relies on time only. Can we do better [qualitative]?)
#2a: show video of select exercises

#3: Approach
#3a: Use classification algorithms, but choose such that they give similarity scores
#3b: Obvious choices: DTW (cost) and HMM (log likelyhood). Others: ANN, decision tree: no score just classify. Naive Bayes: might work (probability)
#3c: This is basically gesture recognition
#3c-1: Problems: usually related to orientation and speed/amplitude
#3d: But we're solving a different problem because
#3d-1: We know the action a-priori, the amplitude is fixed (you have to perform the action as specified in the test) and we can still use time.
#3e: So this lets us work with the path data not just the acceleration
#3f: DTW on the path
#3g: HMM on the path - novel state space
#3g-1: and most importantly we can fix the orientation

#4: Implementation details
#4a: application
#4b: remote control
#4c: algorithms implemented on phone: DTW and HMM (log-likelyhood)
#4c-1: and 'energy' (current method used)

#5: Results
#5a: similar actions = similar scores
#5b: Show DTW and HMM graphs
#5c: Experiments with weights and 'poor motion'
#5d: W/ weights, not conclusion, with 'poor motion' pretty good results
#5d-1: Energy is inconclusive even for 'poor motion'
#5e: Ex. #5 added after other work to show the method wasn't tuned to ex. 1-4
#5f: 'High-box' tested to look at similar actions. Scores are similar, but we're not classifying, similar actions get similar scores.

#6: Conclusion Future work
#6a: We do a more quantitative measurement of the actions similarity
#6a-1: On an easy to use, inexpensive system (with remote control)
#6b: But our experiment is a bit un-scientific
#6c: So future directions are:
#6c-1: Can existing kinematic captures of patients be used to test our system
#6c-2: And of course if we could test it on a few actual patients.

Graphs

I think I've got most of my graphs made. See here.

Also, the gist of the data collected is that purposely screwing up the motion to emulate a patient causes a large increase in the score. This is good. It doesn't always increase the energy. This is also good since energy is sort of the current measure used, we can say we're better.

Jin: can you make some 3D plots of the paths for some of our 'poor motion' examples?

Otherwise I think we're good.

Data from today

Both the raw data, and the split up data is here

Energy results

The energy results are not very encouraging. Exercise #1 seems to be the most effected. Again, it is hard to know how relevant to a stroke patient these results are.

Normal:
Ex: Energy
--------------
1 168.53
2 66.442
3 94.084
4 483.74

Weighted:
Ex: Energy
--------------
1 221.3
2 57.46
3 56.263
4 205.54

DTW results from weighted test

While not a homerun, the results are interesting, see below. The take-away from all of this is that the weights probably changed our motion in a way the DTW algorithm could detect, but the result is not consistent.

So for exercises #1 and #4, the DTW score is lower for the unweighted. Unfortunately, for the others it is higher. However, for both of those exercises the major motion is lifting of the arm, so perhaps the weights matter more there. Also, we don't know how good of an analogue the weights are, so keep that in mind.

Ex. Avg. Normal Avg. With Weights
------------------------------------
1 2079.35  4877.96
2 5925.08  3834.49
3 20313.27  17464.42
4 15911.03  18382.09

Also, it depends on the person:

Andrew

Ex. With Weights (min,mean,max,(stdev))
---------------------
1 1654.56, 2404.40, 3048.16 (438.37)
2 378.62, 957.23, 2641.28, (657.71)
3 17518.23, 26805.36, 41087.51 (5961.64)
4 7375.40, 17831.87, 32354.52 (8689.43)

Jin

Ex. With Weights
---------------------
1 1731.9, 3284.3, 4548.3 (771.31)
2 3947.8, 5788.8, 9190.8 (1820.9)
3 4298, 10568, 20295 (4663.9)
4 4806.2, 9506, 14470 (3355.6)

Scott

Ex. With Weights
---------------------
1 2464.1, 8945.2, 20658 (6588.5)
2 2658.5, 4757.4, 8741.8 (1936)
3 8992.3, 15020, 19028 (3275.4)
4 14906, 27808, 46056 (10216)

Smoothness...

I've done a lot more poking around in MATLAB, and I think that the energy is the best measurement we can use right now. I may post a few plots of the DFT of some of the signals later, but my observation is that it is hard to quantify which parts of the signal are necessary movements and which aren't (the jerks or stop/starts).

However, the energy is a simple measure of the amount of effort required to perform the action. A smooth, quick action will have only the minimal energy required to perform the action, while a slow, jerky action will have much more total energy.

The energy for a signal like this is just the integral of the signal squared, which is closely approximated by the inner (dot) product of the signal with itself. So the following code will give us our smoothness score:

double x_e=0.0,y_e=0.0,z_e=0.0;
for(int i=0;i < A.size();i++)
{
 Double[] sample = A.get(i);
 x_e+=Math.pow(sample[X],2);
 y_e+=Math.pow(sample[Y],2);
 z_e+=Math.pow(sample[Z],2);
}
double t_e = x_e+y_e+z_e;

'A' must be the ArrayList of the acceleration signal. Not the double integrated path. You also need add the following somewhere in the same scope:

private static final int X = 0;
private static final int Y = 1;
private static final int Z = 2;

Path based DTW in Java

I've uploaded a new version of my Java code. I overwrote the previous version.

Once you have the samples in an ArrayList of the x,y,z triples say A and B, then all you do is:

pA = acceltopath(A);
pB = acceltopath(B);
cost = pathcost(pA,pB);

My java code produces exactly the same numerical answer as MATLAB for the examples in the code.

Now I will go about finding the "golden" examples to store in the Android application. Will probably be one of the averages...