study every day 💻📝📚

Session spent cutting up more pilates videos for training data and playing around with the clip length the model trains on. Have been getting some rubbish results on recall for roll-up so focused on adding more roll up videos to balance the data set. Based on my limited experience 5s clip length seems to be best. Kind of annoying that in a lot of the YT videos the movement is done slowly while some explanation is being given

First month of using my work out tracker done. Really like the UX, still get to record my notes by hand but now can run analysis on my work outs.

Session spent working with claude to debug another failing pull up count video, and kicked off a createml training run on my pilates videos. Probably going to pull some videos off YouTube to build up a bigger regression suite for the pull up counter. Will also add some warnings to the app to help improve accuracy eg "camera not vertical enough", "low pose confidence"

Session spent getting claude to fix an issue with my pull up counter app. Had my mate test it out and it heavily under counted. Claude managed to fix the heuristics to get the count accurate. Though I've uncovered some discrepancies between the test script and the app code (the app can drop frames while the cli script wont) so not 100% sure this fix will work. Really like the output from this new version of claude, the table of results was a nice touch

Session spent watching this DHH interview youtube.com/watch?v=JiWgKR… while chopping up pilates videos for more training data. Will see if introducing a new class reduces some over confidence issues. Once the new model is trained will decide whether to tweak it further or move onto the form feedback piece. Also got my pull up counter app working in test flight.

Spent the bulk of the session struggling with deploying to testflight

Session spent deciding on next steps with pilates classifier. Currently leans too much towards roll ups. Confidently categorises lying flat as a roll up. Will introduce a new category "pelvic curl" and see if doing reduces roll up over confidence. If not will have to take a look at the roll up data.

TestFlight is such a faff! Have to get the build approved so I can generate a link to the app. Validation errors on save but no messages. Frustrating! Site is surprisingly buggy for a trillion dollar company. Had one validation error that would appear for 1 second after clicking save and then vanish.

Another successful pull up counter test today!

Session spent working on pilates detector. Got claude to use the script to build a POC iOS app. Filmed some test footage in the garden, worked pretty well. Need to add some more "uncategorised" footage as the model is currently to reluctant to label frames as "uncategorised". I've never done a roll over or roll up before! Will increase the "uncategorised" training data and start using the angles information to provide feedback on the movement. Not sure what the angles should be for correct form, though I'm sure GPT / claude will be able to take a good guess

Session spent getting claude to wire my create ml model into my video processing script and write the label to the output video. Seems to work well on the handful of videos I've tests. Now getting claude to write a tutorial for me on how this wiring all works (I'm pretty novice with swift)

Get a cli script working first then have claude base an iOS app off the script feels like a good approach for this exploratory work. Much quicker iterations.

Session spent adding joint angle calculations and displaying them to my pilates classifier project. Noticed the pose skeleton has low confidence on a bunch of frames so added debug output too. This will be handy for claude to measure it's progress in fixing this issue (when I get round to it). Next up will be using the createML model I trained to display the predicted exercise label of the video.

Session spent playing around with a refactor to avoid a string comparison bug we ran into today. Some business logic checks the string is a specific value and we changed the string without updating this. E2E test would have caught it but feels awkward to rely on them. Think code is missing some value objects.

Will build a cli script first to get it all working together before working on the iOS app

Session spent seeing how my pilates classification model training went in createml (ok), then deciding where to take the project to do something more interesting. Going to have it give basic feedback on how well the exercise was performed. Planning on using a heuristics based approach as that has worked well so far in pull counter app. Thinking use my classifier to detect the exercise then look up heuristic rules for that exercise and check against them, eg see if knees are being in a roll up.

Session spent mostly waiting on createML to extract features from my pilates data set. Read some articles while waiting. Managed to get the pull up counter working on phone today. Pleased! Has two modes: tap record for count overlay, press and hold record for debugging. The app saves the video with overlay to the device, though I thought including the phone in the shot looked better. Will keep testing it whenever I'm on back day. If approach looks good, I'll attempt to add support for other exercises. Considering pulling some exercise videos from youtube to check how robust the heuristics are.

Faint 0 in the top right is the rep counter from the app, main overlay is from the script.

Session spent getting claude to fix some issue in the pull up rep counter app. It detected zero reps, though the script it's based off correctly detected all 15 reps (attached video). Also started setting up a create ML run on my pilates data. However running into an error with loading some of the vids, investigating!

All my other exercise detection experiments have been on video I've filmed myself which is fairly consistent (roughly same angle, lighting, background etc). Interested to see how the diversity of the footage I've pulled off youtube changes things.

Plan is finish off chopping up the remaining roll-over clips (not too many to go) then use create ML to train a model to recognise between three categories, roll over, roll up, and "other". Once this appears to be working will add in new exercise categories.

Session spent getting claude to build an iOS app around the pull counting script we got working in the previous session. Have the app built on my device, will test it tomorrow. As usual spent the downtime chopping up pilates videos into training data.