
Aman
12 posts









Learned something very interesting today! Random projections of a non-linearly separable data onto high dimensional spaces is enough to make it linearly separable. Consider a dataset like XOR that you can't linearly separate. Now, if you project each 2D point onto a D (=50) dimensional space using *randomly* initialised basis vectors, each direction creates a tiny difference between the classes (e.g. gives 51-52% accuracy) because expectation of two classes differs slightly when randomly projected. So each randomly projected feature becomes a tiny discriminator and when you aggregate it over 20-50 such discriminators, a linear classifier is able to separate them perfectly by simply learning how much to weigh each feature. One intriguing possibility of this is that we're able to train deep networks because random projections make most of the data already separable, making the job of gradient descent easy.

Intelligence is on tap now so agency is even more important


Gemini Nano Banana Pro can solve exam questions *in* the exam page image. With doodles, diagrams, all that. ChatGPT thinks these solutions are all correct except Se_2P_2 should be "diselenium diphosphide" and a spelling mistake (should be "thiocyanic acid" not "thoicyanic") :O