Sertaç Özercan

An open-source YouTube Music client for macOS built with Swift and SwiftUI, featuring Liquid Glass UI, lyrics, background audio, and on device Apple Intelligence alternativeto.net/software/kaset… @sozercan

YouTube Music client for macOS github.com/sozercan/kaset/

✨ Join the Azure Upstream Security team in Australia! 🇦🇺 We’re hiring software engineers from early career to senior for roles in #DevTools and #AI to enhance #container security. Be part of innovative #cloudnative, #kubernetes and #opensource projects! #Australia" target="_blank" rel="nofollow noopener">github.com/Azure/containe…

Structured Authentication Configuration is beta in Kubernetes v1.30. Check out the @kubernetesio blog kubernetes.io/blog/2024/04/2… to learn more about the feature. We would love hear your feedback!

@daniel_nguyenx The thing about Arsenal is, they always try to walk it in

Me: My startup is profitable now. 100% bootstrapped. My non-tech friends:

@cpuguy83 That and for future self

Docs are so when people complain "why isn't this written down somewhere?" you can invalidate their ire with a link.

#Llama3 is out! 🦙🦙🦙 Get started to run with AIKit easily with or without GPU, without additional tooling, and #Kubernetes support: 𝗱𝗼𝗰𝗸𝗲𝗿 𝗿𝘂𝗻 -𝗱 --𝗿𝗺 -𝗽 𝟴𝟬𝟴𝟬:𝟴𝟬𝟴𝟬 𝗴𝗵𝗰𝗿.𝗶𝗼/𝘀𝗼𝘇𝗲𝗿𝗰𝗮𝗻/𝗹𝗹𝗮𝗺𝗮𝟯:𝟴𝗯 For more info 👉sozercan.github.io/aikit/

@teknium @maxxrubin_ @NousResearch I was going to report this. Switching to a different quant resolved it

@maxxrubin_ @NousResearch The gguf is broken we are reuploading

We've just uploaded a GGUF of the 8b llama-3 instruct model on @NousResearch's huggingface org: huggingface.co/NousResearch/M…

@perilli @simonw This exists with github.com/sozercan/aikit! ✨

@simonw Partially related: I'd love to see ready-to-deploy Docker images for all these LLMs with nothing but the API entry point. You can set it up one by yourself, but wouldn't be great if we had a library of those to choose from for endpoint deployments?

Which of the LLM API providers have the new Mixtral model?

My team @Microsoft in @Azure is hiring for a Sr. Engineer role to help build novel hypervisor isolated functions using Hyper-V, KVM, and WebAssembly. Most of the work you'd do would likely be in Rust, a little C, and probably a bit of assembly. Sound fun? jobs.careers.microsoft.com/global/en/job/…

🎵 AIKit now offers an extensible solution for finetuning LLMs! Thanks to @UnslothAI , you can finetune fast and efficiently. Then, deploy seamlessly with AIKit using @LocalAI_API for an end-to-end solution! 🚀 Get started sozercan.github.io/aikit 🎬 Demo youtu.be/FZuVb-9i-94

@shanselman @adymitruk @JeremyCMorgan @kubuntu you can do it with a single line 😄 docker run -d --rm -p 8080:8080 ghcr.io/sozercan/llama… check out github.com/sozercan/aikit

@adymitruk @JeremyCMorgan @kubuntu sure, it's a line or two regardless docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama docker exec -it ollama ollama run llama2

LIVE on Youtube - How to run a local AI chatbot on Windows in 5 min, no cuts, no edits, with Ollama, LMStudio, OpenAI youtube.com/watch?v=_AxXtX…

@ibuildthecloud @AlexJonesax I would consider: - change cadence to be less frequent - consider not merging until before release - choose what you want to merge like security fixes - group relevant updates together like k8s.io deps or GHA - consider auto merge if you are confident with CI

@AlexJonesax Do you have a solution? I hate this. I just stopped enabling renovate.

This is 90% of OSS maintainers time spent

Spent the day looking at Project Copacetic and my mind is blown! It works so well, and everything I have thrown at it has worked and patched. Going to get this into our deployment pipelines asap! Take a look! #CloudFamily #Kubernetes github.com/project-copace…

🚀 Azure Upstream team is hiring! Looking for Principal & Jr devs to build cloud native OSS projects with #Kubernetes & CNCF. 🌍 Work remotely in Czech Republic, Serbia, Poland, Romania. ✉️ DMs open! jobs.careers.microsoft.com/global/en/shar… & jobs.careers.microsoft.com/global/en/shar… #hiring #opensource

🏗️ AI + BuildKit = AIKit: Build and deploy large language models easily Powered with @Docker github.com/sozercan/aikit

@patloeber Check out github.com/sozercan/aikit too! Uses @LocalAI_API and does not need any tooling but @Docker!

Want to run LLMs locally on your Laptop?🤖💻 Here's a quick overview of the 5 best frameworks to run LLMs locally: 1. Ollama Ollama allows you to run LLMs locally through your command line and is probably the easiest framework to get started with. Just use the installer or the command line to install it. Then you can type `ollama run modelname` and it starts an interactive session where you can send prompts. It supports all important models like llama2, mistral, vicunia, falcon, and many more. And trying out new models is as easy as running `ollama pull modelname`. 🔗github.com/jmorganca/olla… 2. GPT4All A free-to-use, locally running, privacy-aware chatbot. This is kind of a ChatGPT clone that comes with a nice UI and installers for every major operating system. You can also download embedding models and upload local documents that the model can use to retrieve information. 🔗github.com/nomic-ai/gpt4a… 3. PrivateGPT Similar to GPT4All, PrivateGPT also comes with a nice UI to chat with LLMs. Its focus does not lie on trying out many different models, but rather on interacting with your own documents 100% privately. It provides a nice @Gradio frontend where you can easily upload your files and then query the documents. 🔗github.com/imartinez/priv… 4. Llama.cpp LLama.cpp, created by @ggerganov, is a port of Facebook's LLaMA model in C/C++. This is probably the goat of all local LLM frameworks and to my knowledge was the first project that allowed to run LLMs easily on a MacBook.🐐 Today, it not only supports the first Llama model but also all other major LLMs. It’s also worth mentioning that thanks to this project there is a new model format - GGUF - that is used in all previously mentioned frameworks, too. So this project enables the other frameworks. It is a bit more tricky to get this running since you have to clone the repo and build it from source, and also have to obtain the model weights separately and run some scripts to convert it to the correct format. The easiest way I’ve found to get started is to download these already converted and quantized Llama 2 models from @huggingface, thanks to @TheBlokeAI: huggingface.co/TheBloke/Llama… 🔗github.com/ggerganov/llam… 5. LangChain @langchain LangChain is a framework for developing applications powered by LLMs and is not focused solely on running LLMs locally. But among its many features it also provides a whole guide about running LLMs locally. It shows how to import Ollama, Llama.cpp, and GPT4All into Langchain to build more complex applications on top of it. This approach involves more coding but it also offers the most flexibility. 🔗#llms" target="_blank" rel="nofollow noopener">python.langchain.com/docs/guides/lo… --- If you enjoyed this thread, I also have a video version with a short demo of every framework: youtu.be/5WCvGyPpWwg

Want to build and deploy your own #chatgpt locally or to #kubernetes on any cloud? Introducing AIKit! 🤖✨ no need for GPU or other tooling. All you need is Docker! github.com/sozercan/aikit

@jrrickard @cpuguy83 is there an api? can we get ai to generate tweets?

@sozercan @cpuguy83 We improved ours by adding eyes to it, so when it crashes it’s even funnier

Today the roomba didn't even make it off the charging station before it crashed (as in the software).