Armaan

after a year of procrastinating, my personal website is finaallllyyy live✨ it only took me 3 evenings after work to put it all together... should’ve done this way sooner!

been reading a lot about RL and policy collapse lately and i can't stop thinking about the analogy in real life — what happens when we all optimize for the same reward function wrote some thoughts here: @sandhuapk/what-are-we-leaving-behind-for-museums-6615bc536bc7" target="_blank" rel="nofollow noopener">medium.com/@sandhuapk/wha…

@demonshadow007 started with this: youtu.be/zduSFxRajkE?si… and then read up more on the mentioned references here

@apkinverse From where are you reading? Any suggestions?

reading more about tokenizers has made me such an empath. no one should have to deal with my spelling crimes.

@__HorizonX__ agree, i think it comes more easily with practice, but requires deliberate effort in the beginning

@apkinverse "figuring out what i actually wanted from a paper" is such an underrated skill. most people sit down with a paper with no real question in mind, just hoping the paper will tell them what matters. that's where hours disappear.

feb × papers wrapped - i tried reading research papers every day in feb. the goal was: compress exposure, get better at reading papers, and stay closer to what’s happening in current research. what i gained/what was hard 👇

some more papers i read but didn’t post about: - Generative Adversarial Imitation Learning - Efficient Probabilistic Performance Bounds for Inverse Reinforcement Learning - Maximum Entropy Inverse Reinforcement Learning - Prompt Optimization Makes Misalignment Legible

what was hard: - resisting the urge to treat it like a checklist - figuring out what I actually wanted from a paper - intuition, math, implementation, or just the big idea - accepting that some papers need multiple passes - being consistent

Imagine being this early

@tensorcruncher @Cohere_Labs hi yes, thanks. i am already on it <3

@apkinverse Check out the expedition tiny aya by @Cohere_Labs starting today

internship search update:

silicon valley was a documentary damn it jian yang

4/ Both need some form of human data: IRL assumes demonstrations encode the true objective. TAMER assumes feedback encodes the true objective.

3/ they also differ in temporal structure. IRL preserves long-horizon reasoning: infer reward -> run RL -> optimize expected return. TAMER removes credit assignment: predict immediate approval -> act.

1/ i read about TAMER and IRL this week - both try to handle reward misspecification, which can make them seem similar. but the core difference: IRL tries to recover the "goal" behind behavior. TAMER models "human approval" signals directly.

6/ but inferring the “true” objective from behavior is ambiguous as multiple reward functions can explain the same behavior. also, IRL problem needs assumptions (e.g. linearity) and good features. without that, the recovered reward might just be an artifact. this makes it tricky

5/ compared to behavior cloning: behavior cloning -> supervised learning (state -> action) fragile to distribution shift IRL -> learns why the expert chooses actions then plans accordingly this can avoid cascading error problem in BC.

days 14 and 15 of feb × papers read about Apprenticeship Learning / Inverse Reinforcement Learning (IRL) - learning the objective from expert behavior rather than hand-crafting reward functions. what i read 👇