Mark

New kernel post! This time: /proc/*/mem and how it writes to unwritable mem. Key takeaway: By walking page tables in software, the kernel can access userspace mem without directly dereferencing pointers it gets from userspace. (No WP/SMAP bits involved). offlinemark.com/2021/05/12/an-…

fixed saving/restoring rflags, also correctly enabled/disabled interrupts during the scheduler critical section

week 111, streaming kernel dev right now topic: kernel threads (cooperative) continuing with scheduler work today, will fix some interrupt-safety topics and generally continue this line of work see ya 👋 youtube.com/live/VRukq7L1F…

@Julian love seeing this vision come to life!

i live on ranch, and it cost me less than buying an apartment in california. the downside is isolation from city life. if that doesn't bother you, quality of life shoots up. you'll become obsessed with animals 😂 i have 2 mini cows, mini horses, poodles. must stop, but... learning to train them is too fun. i've included a video of my mini cow giving "hello friend" head nods to my new mini horse i'm also building a soundshed on the property with two hifi audio systems to incept my friends to fall in love with the hobby. over time, you also become interested in being off grid. think solar, starlink, rain catchment, cattle + chickens, herb + fruit + veggie garden. life slows down and you start to value building these skills—to self-sustain. going outside between Zoom calls and just looking at the trees and hearing wind rustle through them is so calming. this is how i grow up. shouldn't have done a full 15 years in the city in between. i've built a bunch of guest cabins, and friends can come stay for a couple weeks at a time. i put guitars + pianos + hifi in their cabins so they can feel creative next up is a giant golf cart racing course that traverses through the property

week 110, streaming kernel dev right now topic: scheduling (cooperative) last week we got basic cooperating scheduling between two kernel threads working, will continue with that this week including investigating some weird bugs see ya 👋 youtube.com/live/9eH4k3qAZ…

@vector35 Too kind❤️ appreciate you!

@offlinemark Even if you weren't, people should check out your streams! Great resource for anyone into OS internals.

week 109, streaming kernel dev right now topic: kernel threads (cooperative) last week, we started a basic scheduler/thread implementation and got a single kernel thread running. going to work on switching between threads today see ya 👋 youtube.com/live/wWDHk4no9…

also very proud to be using @vector35 Binary Ninja on stream =)

i opted to copy all the general purpose registers into a per-cpu global scratch area. this frees us registers for immediate use, allowing to do things like read the RIP value from the stack, and compute the RSP value to store then after entering back into C++ and obtaining the pointer to the kernel, I can locate the current task structure and copy that state back in not sure if there's a better way, but it's how i'm doing it. the reason for using the scratch area instead of the stack is to prevent accidentally overflowing off the end of the stack. the global scratch area is safer in that regard

@MaxwellChegbe ty maxwell!!

@offlinemark Bro this is some insane consistency! I always follow on YouTube!

prototyped basic kernel thread scheduling

week 108, streaming kernel dev right now topic: object oriented kernel going to step back from the virtual memory and interrupts prototyping to do a bit of software engineering. gonna play around with using oop in the kernel see ya 👋 youtube.com/live/WbnP_HLQL…

week 107, streaming kernel dev right now topic: interrupts continuing with interrupts after getting GP fault and basic software interrupt (INT instruction) to work last time see ya 👋 youtube.com/live/1X_eqapSc…

week 106, streaming kernel dev right now topic: inline asm / interrupts continuing with converting att syntax to intel, and then moving forward with setting up handlers for other kinds of interrupts/exceptions see ya 👋 youtube.com/live/F2iOR1MZC…

@nsg650 Brain not big enough for that yet

@offlinemark Add c++ exception support next /s

week 104, streaming kernel dev right now topic: x86_64 interrupts last week we started interrupts and got the most basic divide by zero exception handler working. will continue and flesh it out with a trap frame and handling it in c++ see ya 👋 youtube.com/live/h6HwJDy0n…

got a simple divide by zero exception being handled

week 103, streaming kernel dev right now topic: x86_64 interrupts i'm moving on to setting up interrupts. i already did this before on JOS, but now will revisit it on 64 bit to learn how that differs, including new concepts like the IST see ya 👋 youtube.com/live/vVzgOEaWu…

We've been working our asses off on Sonic Mania for the Sega Dreamcast. @SonicFreak94, our team leader, has been redoing the allocator to conserve RAM and jnmartin has been implementing the missing draw routines for 3D geometry... What about me? I got ONE JOB. I'm the dude who is tryna make sure the 3D stages don't run like absolute shit and make the Dreamcast look weak in front of the world! We already moved all 2D and 3D rendering to the DC's PowerVR GPU, which can push a bazillion times as many polygons as the 3D stages use, so there should be no problem handling them, riiiiight? NOPE. It's not that easy! Our poor SH4 CPU is now being absolutely murdered and under-utilized due to the fact that all of the matrix and vector math for the 3D graphics transforms has been implemented using FIXED POINT INTEGER OPERATIONS... which means the SH4 doesn't get to utilize its baller FPU or special vector instructions that are responsible for accelerating this kind of math... and more importantly, it means none of these structures are compatible with my SH4ZAM linear algebra library we use to accelerate math for gainz on most of our Dreamcast ports... So what do we do? We can't just hand-convert the bazillion different matrices throughout the engine and game logic into floats, when all of their initialization constants and surrounding math assumes they are represented as fixed-point integers. That would take a million years and be too error-prone... So after wallowing in self-defeat for a few hours and questioning whether it was even possible to accelerate this shit reasonably... I came up with what I think is an ingenious solution. We're going to replace the original Matrix struct containing a 4x4 array of fixed-point int32_t entries with some crazy, newfangled C++23-based abomination that leverages some questionable and evil language features, allowing it to be represented as a 4x4 array of floats internally... and we're going to do it noninvasively, so that the rest of the game and engine code, which operates on fixed-point integers, will still continue to work properly without needing to change a line of client code... SO LETS DO IT! The fundamental lowest-level construct which enables this behavior is the "Scalar" class, in the top-left pane. It simply encapsulates a single floating-point value internally, but it provides a series of overloaded conversion, assignment, relational, and arithmetic operators, which allows the evil bastard to masquerade as though its still a fixed-point integer! So the old code can continue to assign fixed-point values to it and do intermediate fixed-point arithmetic on it, but the resulting values will internally be converted to and stored as floats, allowing us to use our FPU on them later. Then one layer of abstraction above the "Scalar" class, we construct our "Matrix" struct in the right pane, which exposes a 4x4 array of "Scalar" elements along with various methods implementing its transform operations. The secret sauce here is the "shz()" methods, which allow a Matrix instance to be interpreted as a shz::mat4x4, which is the 8-byte aligned, 4x4 floating-point matrix which is compatible with my SH4ZAM library's C++23 API. This allows us to simply implement our Matrix transform methods by simply forwarding them on to SH4ZAM's hand-written SH4 assembly-optimized routines! Finally, in the bottom left, you can see how transforming a 3D vector by a 4x4 matrix went from some gross-ass integer-based, bit-shifting monstrosity into something adorable, which will ultimately become a single instruction FTRV under-the-hood for massive gainz on the most taxing operations, the 4x4 matrix multiplications and 4D vector transforms!