R. James Cotton
Physiatrist, neuroscientist, gait and movement, brain and robotics enthusiast. Assistant Professor at Northwestern and Shirley Ryan AbilityLab
- I want to like Gemini 3 but gemini-cli is the most enraging, terrible experience. It is so impulsive and while trying to develop a plan it just starts making random half baked code edits. No amount of agent file or memories seems to prevent this. It is unaligned and usless for anything complex.
- In contrast Claude Code is a tougher because of the smaller context window, but ultimately does what I want (less impulsive), including implementing complex plans through a series of subagents. It feels like the inherent forgetfulness of the subagents keeps it much more aligned on the big picture.
- At #SfN and want to hear about how BiomechGPT can provide a language interface to your biomechanics data? Checkout poster happening now at YY4 with Ruize Yang and @antihebbiann.bsky.social
- If you are at @acrmrehab.bsky.social I will be presenting this morning (8am) SPECIAL SYMPOSIUM: Ai-powered Movement Analysis and a Causal Framework for Precision Rehabilitation SS1 #ACRM2025 cdmcd.co/5npwMR
- Super proud of this work led by JD Peiffer (@abilitylab.bsky.social and @tgsatnu.bsky.social) preprint: arxiv.org/abs/2507.08268 project/code: intelligentsensingandrehabilitation.github.io/MonocularBio... Open source code that lets you get state-of-the-art biomechanics from a smartphone
- We developed a novel approach to fitting biomechanics from smartphone video that produces kinematic reconstructions within a few degrees and has been validated across a wide range of activities and clinical backgrounds.
- Central to this was extending our end-to-end differentiable biomechanics approach to fitting both 2D and 3D keypoints measured from images. This can also account for smartphone rotation measured by our Portable Biomechanics Platform, which also makes this easy to integrate into clinical workflows.
-
View full threadMore demos and code available at intelligentsensingandrehabilitation.github.io/MonocularBio... JD did a great job creating a Gradio demo so try it out and let us know what you think And here is a video of JD going on a celebratory run that the preprint is out :)
- Enjoyed presenting on "The Good, The Bad, and the Ugly: AI for SCI Clinicians" with @ryansolinskymd.bsky.social and @josezariffa.bsky.social. Great enthusiasm from the crowd on the topic and the lively discussion and nice followup from precourse bsky.app/profile/ryan...
- AI integration in spinal cord injury medicine precourse at the ASIA2025 meeting. Led by @peabody124.bsky.social and Dr. Sarah Brueningk. Learning lessons from other successful examples in Cancer, Alzheimer’s, Cardiology.
- Reposted by R. James CottonAI integration in spinal cord injury medicine precourse at the ASIA2025 meeting. Led by @peabody124.bsky.social and Dr. Sarah Brueningk. Learning lessons from other successful examples in Cancer, Alzheimer’s, Cardiology.
- Over the last few years we have been developing methods for markerless motion capture of biomechanics and getting them into the clinics, such as at @abilitylab.bsky.social. We are now developing foundation models from these large datasets and testing what this enables. Two recent preprints:
- The first is KinTwin arxiv.org/abs/2505.13436 which trains torque driven and muscle-driven policies to replicate movements of intact and impaired gait. It detects clinically meaningful features like propulsion asymmetries and muscle timing. Teaser from a few months back: bsky.app/profile/peab...
- We already do, although currently just for the bilevel scaling and marker offsets that we optimize jointly with the inverse kinematics (arxiv.org/abs/2402.17192) Starting to think about scaling for muscles, which we need for some of our new stuff to get rid of the scaling inconsistency.
- Since then, we've tuned it up to handle anthropomorphic and muscle scaling. Still lots of work to do further tuning this as there are many things we aren't scaling such as mass and inertia and optimizing w.r.t. the EMG data we have from our wearable sensors.
-
View full threadLooking forward, we hope to combine things like this. E.g. using BiomechGPT to understand movement user requests and then run simulations in the physics simulator using imitation learning, for example. Either way, really starting to see promise for foundation models in biomechanics Stay tuned :)
- Finally recovered from RehabWeek in Chicago, which was fantastic. It was a very successful week for the Intelligent Sensing and Rehabilitation lab at @abilitylab.bsky.social. (p.s. sorry below for anyone on BlueSky who I couldn't find/failed to tag)
- With Dailyn Despradel, Derek Kamper, @marcslutzky.bsky.social, and @dougweberlab.bsky.social we organized a workshop on EMG biofeedback. Very much enjoyed the engaged discussion on how to disseminate these technologies into the real world.
- With Kyle Embry and the @abilitylab.bsky.social C-STAR team we also organized a half-day workshop "From Motion to Meaning: AI-Enabled Biomechanics for Rehabilitation" showcasing work from Georgios Pavlakos, Eni Halilaj, Vikash Kumar, Chris Awai, and Pouyan Firouzabadi.
-
View full threadIt was also the initial meeting of Julius Dewald and Bob Sainburg's Society for Neuromechanics in Rehabilitation (SoNMiR) and it was great to present on biomechanics in rehabilitation and arxiv.org/abs/2411.03919. Very exciting for this society bringing together this community.
- Looking forward to presenting on what we can do with large-scale biomechanics data in rehabilitation in the SoMNiR #RehabWeek 2025 session this afternoon! @abilitylab.bsky.social
- @jdpeiffer.bsky.social and Tim Unger gave a great talk in the ICORR best student paper session on their work using markerless motion capture to track arm kinematics of people with stroke. arxiv.org/abs/2411.14992
- Really enoyed the session and engagement!
- We're enjoying an engaging Q&A session with our #ASNR2025 speakers Dr. @lenating.bsky.social, Dr. Lucas McKay, Dr. Trisha Kesar, @peabody124.bsky.social, and Dr. Hyeok Kwon. We appreciate these thoughtful questions from the audience and responses from the panel! #neurorehabilitation #neuroscience
- Reposted by R. James CottonContinuing our #ASNR2025 symposium on #ArtificialIntelligence and computational modeling in #neurorehabilitation, @peabody124.bsky.social discussed how his lab is quantifying gait-based biomarkers during functional mobility tasks in diverse clinical settings. #AI #neuroscience #rehabilitation
- Reposted by R. James CottonVery much looking forward to our speakers: @maryamshanechi.bsky.social, @hugospiers.bsky.social, Marom Bikson, @jenpitt.bsky.social, Mary Czerwinski, @peabody124.bsky.social, @dpferris.bsky.social, @sladouce.bsky.social, Matthew Rizzo, @c-rothkopf.bsky.social, @doriswang.bsky.social, Ying Choon Wu
- Reposted by R. James CottonDon't miss our #ASNR2025 session on #AI and computational modeling to diagnose and treat functional mobility deficits! We can't wait to hear the latest from Drs. @lenating.bsky.social, @jlucasmckay.bsky.social, @peabody124.bsky.social, Trisha Kesar, & Hyeok Kwon! 🧠🔄 🧠📈 🧪 www.asnr.com/i4a/pages/in...
- @ben-petrie.bsky.social gave a great #physiatry2025 presentation on EMG biofeedback for people with spinal cord injury and won first prize! Thanks to the Craig H. Neilsen Foundation for support for this project, and to all of our participants. Excited to share this soon in an upcoming manuscript!
- Are you interested in data science and spinal cord injury? Looking for an opportunity to explore it? Then I highly recommend checking out the data science challenge organized through ASIA. www.kaggle.com/competitions... www.kaggle.com/competitions...
- I'm at APTA CSM: - (PD-21072) Pediatrics Research Forum Precision Rehabilitation 02/13/25 Time: 3:00PM-5:00PM - (NE-21770) Markerless Motion Capture Tools for Outcomes Assessment: Expanding Capacity and Diversity for Next Generation Neurorehabilitation. 02/14/25 Time: 11:00AM-1:00PM
- Pretty pleased about this work arxiv.org/abs/2502.06486 with @sinzlab.bsky.social showing how we can infer kinematic confidence intervals from multiview markerless motion capture. At least for our purposes, methods like this are equally important to traditional validation studies. #BiomechSky
- Super stoked about the progress @jdpeiffer.bsky.social is making on monocular biomechanical analysis. Also really fun to connect back to my old drone hobby (and they are so much easier now than a decade ago buildandcrash.blogspot.com/2014/10/refl...)
- Another fun one. In realtime (instead of the 60 fps -> 30 fps in the prior)
- The team did great work validating end-to-end optimization of markerless motion capture in the upper extremity for people with stroke. Compared to optical motion capture, tracking the elbow within a few degrees :). Other metrics are also great. Really enjoying the collaboration with LLUI!
- Tim Unger, Arash Sal Moslehian, J. D. Peiffer, Johann Ullrich, Roger Gassert, Olivier Lambercy, R. James Cotton, Chris... Differentiable Biomechanics for Markerless Motion Capture in Upper Limb Stroke Rehabilitation: A Comparison with Optical Motion Capture arxiv.org/abs/2411.14992
- Reposted by R. James CottonWe've started working on a #Neurorehabilitation Starter Pack to help researchers, clinicians, and other professionals in the field get connected. Please reply or message to be added to or removed from the list. We're excited to see a growing neurorehab community here! go.bsky.app/4Vgz8Kzat://did:plc:zq5iz4hfj6x4w6a4kjofv3rn/app.bsky.graph.starterpack/3layv66ixlr2f
- Reposted by R. James CottonThis starter pack is intended for researchers in all areas of gait and posture - just let me know if you would like to be added. go.bsky.app/UaoaLRZat://did:plc:w4lveenpjymzrvadn4rjcuxr/app.bsky.graph.starterpack/3lb5xeu3zmx2c
- Been having some great conversations and presentations about this preprint since it came out. I'm really pleased by how postively many people are responding to it. I'd love to hear from the community of any people they know doing relevant causal inference work (rehab or not).
- While rehabilitation is moving towards an era of big data, we still lack a framework to analyze this data to improve outcomes. We took a stab at this, which we call a "Causal Framework for Precision Rehabilitation." arxiv.org/abs/2411.03919
- Also to kickstart the list I'm going to put out there this work from Kat Steele's group: www.sciencedirect.com/science/arti...
