Cute moloch

@joepalerm0 strategies (which questions to ask/what tools to use to extract the exact citations for each question. Will the model generalize better when you train it on the synthetic dataset where the creativity is embedded in the step by step system usage? Intuitively it makes sense.

@joepalerm0 Besides training on the base model with specific domain knowledge what would you say about: 1. Build a specialized information retrieval system with several tools. 2. Create synthetic dataset on how system is used to retrieve expert knowledge. 3. Train on step by step retrieval..

"The models just want to learn" -Ilya Sutskever

If you believe OpenAI is not open you are simply not paying enough attention