Mark

New kernel post! This time: /proc/*/mem and how it writes to unwritable mem. Key takeaway: By walking page tables in software, the kernel can access userspace mem without directly dereferencing pointers it gets from userspace. (No WP/SMAP bits involved). offlinemark.com/2021/05/12/an-…

@EricccTaiwan Eric! I’ll DM you :)

@offlinemark Hi Mark! Welcome to Taiwan, when r u going to stop by 😃

hello friends! anyone by chance have connections to the hacker community in Taipei/Taiwan? Touring around for a bit and would love to connect 😊

@anthony_nj23032 Ty zuko

@offlinemark your consistency is epic 💪

week 116, streaming kernel dev right now topic: kernel threads today i'll be debugging a boot loop issue that appeared after I cranked up the number of tasks on the system from 4 to 4000. in general working more on scheduling and refining it see ya 👋 youtube.com/live/UrVVFu81Y…

@Shiftreduce @h0mbre_ i've heard that's hard

@offlinemark @h0mbre_ can't wait to see you develop a truetype parser

ok. the problem was caused by 1. having guard pages 2. unmapping them all from the VA space all on init 3. managing them all + the task stacks in an ETL pool the ETL pool uses a freelist internally to implement the allocator and when an object in the pool is unused, it superimposes a metadata struct at the start of the object with eg the freelist pointers this breaks if I unmap the guard page, since the metadata is no longer writeup too bad, looks like I'll have to find another way to allocate task stacks (i've been trying to avoid writing a general purpose kernel heap)

deferred task cleanup implemented! after the next task is chosen, I switch stacks, run a cleanup routing on the new tasks's stack, then enter the next task

week 115, streaming kernel dev right now topic: kernel threads continue with properly cleaning up task stacks at the end of a task lifetime, probably implementing some kind of deferred task cleanup see ya 👋 youtube.com/live/eFS0hbpKe…

week 114, streaming kernel dev right now topic: scheduling working more on scheduler stuff -> idle task, scheduling policies, mapping the stack, etc see ya 👋 youtube.com/live/ap76LbNwW…

@0xSumit_ Thank you bro

This bro @offlinemark has been building his own operating system from scratch in C++ for 113 weeks straight and is still streaming it live. not a tutorial and not a clone. writing the actual kernel, scheduler, memory management, trap handlers all from zero. that's a different level of obsession fr 🗿

@LearnerKJ lol

crazy geek @offlinemark 🤖

got it working, tasks can yield and exit. idle task ensures there's always something runnable I still don't have a virtual memory allocator, so I punted on the "properly map stacks instead of using physmap" by statically allocating a stack frame directly in the Task struct lol

week 113, streaming kernel dev right now topic: kernel threads continuing with scheduling/kernel threads, working on thread exit, an idle task, and properly mapping stacks instead of using the physmap see ya 👋 youtube.com/live/DM13pqlDe…

got kernel thread exiting working, including a little landing pad if a task accidentally returns without calling the exit API! I prepare the stack such that there is a RIP value at the top that points to a landing pad function that calls the exit API (this is doable since tasks aren't called into, but rather push+ret'd into)

week 112, streaming kernel dev right now topic: scheduling continue on scheduling, refactoring the yield path with an idea to remove the global scratch buffer. and then creating an api to allow tasks to exit see ya 👋 youtube.com/live/Jtw6fD75u…

fixed saving/restoring rflags, also correctly enabled/disabled interrupts during the scheduler critical section

week 111, streaming kernel dev right now topic: kernel threads (cooperative) continuing with scheduler work today, will fix some interrupt-safety topics and generally continue this line of work see ya 👋 youtube.com/live/VRukq7L1F…

@Julian love seeing this vision come to life!

i live on ranch, and it cost me less than buying an apartment in california. the downside is isolation from city life. if that doesn't bother you, quality of life shoots up. you'll become obsessed with animals 😂 i have 2 mini cows, mini horses, poodles. must stop, but... learning to train them is too fun. i've included a video of my mini cow giving "hello friend" head nods to my new mini horse i'm also building a soundshed on the property with two hifi audio systems to incept my friends to fall in love with the hobby. over time, you also become interested in being off grid. think solar, starlink, rain catchment, cattle + chickens, herb + fruit + veggie garden. life slows down and you start to value building these skills—to self-sustain. going outside between Zoom calls and just looking at the trees and hearing wind rustle through them is so calming. this is how i grow up. shouldn't have done a full 15 years in the city in between. i've built a bunch of guest cabins, and friends can come stay for a couple weeks at a time. i put guitars + pianos + hifi in their cabins so they can feel creative next up is a giant golf cart racing course that traverses through the property

week 110, streaming kernel dev right now topic: scheduling (cooperative) last week we got basic cooperating scheduling between two kernel threads working, will continue with that this week including investigating some weird bugs see ya 👋 youtube.com/live/9eH4k3qAZ…

@vector35 Too kind❤️ appreciate you!

@offlinemark Even if you weren't, people should check out your streams! Great resource for anyone into OS internals.

week 109, streaming kernel dev right now topic: kernel threads (cooperative) last week, we started a basic scheduler/thread implementation and got a single kernel thread running. going to work on switching between threads today see ya 👋 youtube.com/live/wWDHk4no9…