Armin Buescher

@lucataco93 Nice! FYI peak memory during training depends on batch size and # tokens in the dataset. You should give SiLLM a try for training with LoRA/DPO: github.com/armbues/SiLLM

Fine-tuning Llama 3 8B on an M3 Max via MLX Running the same example: - Batch size 4, 16 LoRA layers - 411 tps - Avg power 37W - Peak mem 20GB

@DaveDC22 There was a bug that was fixed in the app. Pull the latest source from the repo and it should work 👍

@armbues I figured out how to download the model correctly, but now I keep getting this error in the UI "'str' object has no attribute 'apply_chat_template'

Running Llama-3-8B-Instruct on Mac with the SiLLM framework powered by MLX... just took some fiddling with the tokenizer & template to get it to run 😁

@DaveDC22 You need to point it at a directory with model files that you want to run. What type of model are you trying to load?

@armbues This is amazing. I'm trying out your program, but I keep getting this reponse in the chat. "No weights files found"

@awnihannun I could not agree more! 💯 Fantastic job by the team working on this! 👏

One of my favorite things about MLX is it helps put ML research back in the hands of a single bold hobbyist. Don’t need a supercomputer to invent - just a nice laptop, a vision, and some persistence, (and maybe pip install mlx 😉)

@alew3 @awnihannun An out-of-the-box solution to run/train LLMs on Apple Silicon built on top of MLX: github.com/armbues/SiLLM

@armbues @awnihannun what is the SiLLM framework?

Early version bump for SiLLM to 0.1.1 with some bugfixes and support for Llama-3 models. pypi.org/project/sillm-… github.com/armbues/SiLLM

@adithyan_ai Just loading the model needs about 87 GB and then you’d need a bit more for inference.

@armbues Nice. How much is minimum RAM required?

Running WizardLM-2-8x22B 4-bit quantized on a Mac Studio with SiLLM powered by Apple MLX

@ivanfioravanti @awnihannun Just the product of lots of tinkering and trying to port DPO & losses over to MLX 😁 Might have some bugs that I'm not seeing 🙈 Example code with the DPO-mix dataset here: github.com/armbues/SiLLM/…

@awnihannun DPO? WOW! This is a game changer!

Very cool new MLX project: SiLLM - Fine-tuning LLMs with DPO + LoRA - A nice UI for generating text with different models pip install sillm-mlx Code: github.com/armbues/SiLLM/…

A huge thank you to @awnihannun @angeloskath and the rest of the team for developing the MLX framework that SiLLM relies on! Also big kudos to all the contributors of the MLX Examples project 👏

The repository includes several code examples: - LoRA training with the Nvidia HelpSteer dataset - DPO Fine-tuning with the DPO Mix 7K dataset - Implementation of the MMLU Benchmark - Calculating perplexity scores of a model for a sample dataset

I'm excited to share a new open-source project: the Silicon LLM Training & Inference Toolkit, short SiLLM. Check out the project on Github here: github.com/armbues/SiLLM

One of the reasons to attend the #CARO2023 is the food for thought that is delivered in talks, conversations, and of course keynotes. Armin Büscher @armbues will share his technical perspective about innovation and disruption in cybersecurity. #c228106" target="_blank" rel="nofollow noopener">caro2023.org/#c228106

Y’all have a home on Infosec.exchange if you need it ❤️

This file leaked an Security Enterprise Virustotal API Key before！But now it's expired because someone leaked the key😅 ITW:07c4a75b1422a22ec29c5102e0b67055 API Key:d10468bead05da1685629a0abcfed5f963d6adbc7e6bb2b2fc343dbb36be0349 unbelievable！

We just released 1000+ yara rules and 200+ endpoint behavior rules github.com/elastic/protec…

I'll be traveling to Vegas for #blackhat2022 and #DEFCON next week. Looking forward to hang out with many infosec folks I haven't seen in a long time 🥳