Andrew Toth

@NHpilled We fixed this. x.com/L0RINC/status/…

How dare the Coretards allow running a node on a raspberry pi to be brutally murdered in broad daylight and then have the gall to oppose the effort to stop it from getting even worse. Run BIP-110 to keep noderunning alive!

@reardencode @Ademan555 @fanquake Hmm they both say 62 GB of total RAM. Could be mislabeled on the graph. But, this change now makes higher dbcache values have less impact. So it might no longer be worth it to use a higher value than default.

@Ademan555 @fanquake @andrew_s_toth The machine with the smaller dbcache had more total RAM, leaving WAY more RAM for OS-level file caching.

Congrats to @andrew_s_toth for shipping his parallel input fetching work. Pretty clear impact

@Jethroe111 We fixed this, it won't slow down now. x.com/fanquake/statu…

This is a stupid, short sighted take from a man who has never fired up a node in his office and watched the fucking thing slow down when it hits the spam wall of '23. If you were sitting in an F18 next to me on a carrier I would give you the signal, "Pull your head out of your ass."

@udiWertheimer @Jethroe111 Next release will be over 2 hours faster to sync. So, ordinals have added nothing. x.com/L0RINC/status/…

all of the jpeg data is adding maybe, i don’t know, 2 hours of sync time if i’m being generous? 2 hours that you only do once and never again? is it really worth it for you to spend 5 hours a day, for years now, being upset about an extra 2 hours your node had to process things one time? it’s in the past man, your node already processed it look man, bip110 is not going to happen. i think you probably know that but my company hasn’t put a jpeg on bitcoin in over 2 years. massive companies like magic eden and binance bailed out and removed ordinals support. businesses like ord-io shut down. holders of ordinals lost money and they’re not buying more. what more do you want? is it really worth you still being upset about this on daily basis for the foreseeable future? you only live once brother

@arejula27 @L0RINC There's a lot of info here :) delvingbitcoin.org/t/swiftsync-sp…

@andrew_s_toth @L0RINC I need to read more 😍 I guess the discussion is in delving and mailing list. Another source to take a look at?

🚀 LFG 🚀 @andrew_s_toth’s parallel input prevout fetcher was just merged into Bitcoin Core for the upcoming v32 - after 2+ years and 1000+ review comments. IBD is getting *even* faster \:D/ Thank you for the persistence, Andrew: this is what open-source collaboration should look like! github.com/bitcoin/bitcoi…

@arejula27 @L0RINC Yes, that's called SwiftSync.

@L0RINC @andrew_s_toth Would it be possible to parallelize IBD by processing blocks in batches, where each batch produces a diff to the UTXO set? After validating all batches, the diffs could be replayed to verify that they produce the correct final UTXO set. I remember seeing something on delvinng

@Kruwed @stephanlivera We can fix this! github.com/bitcoin/bitcoi…

@stephanlivera Why is there a big spike in optional indexes in 2036?

Spam arguments are big now, for a bit of fun I made a Bitcoin Node Cost Tracker/Dashboard stephanlivera.github.io/node-cost-dash…

@cryptomeerkat @rot13maxi @L0RINC @zndtoshi This one github.com/bitcoin/bitcoi…

@rot13maxi @L0RINC @zndtoshi This one? github.com/bitcoin/bitcoi…

Would it be possible to add a feature in Core and Knots implementation so that if you set a flag in the config file then the node becomes a different type of pruned node: it starts checking all blocks from the start and the new ones coming in and after they are validated it would delete the part that is "spam" from the node. So thus deletes all the op_return data, what's after op_if and so on, whatever bip110 tries to implement through the fork. From that point on that node can still be used to validate all new wallets (still has transactions history compared to pruned node in Core) but will not provide those specific blocks for others to IBD, just the blocks that are "pure". We can call the flag "purify=1".

@caesrcd Unfortunately a private broadcast connection is different from that normal connection. The pb connection will immediately send the inv, before receiving the pong.

@andrew_s_toth GNOBAN acts well before the inv broadcast starts. I monitored the process and took two screenshots: one showing the peer being filtered, and another showing it not being filtered.

While we wait for the patch: gnoban -t v1 Running GNOBAN with this parameter will filter and ban legacy v1 peers, reducing exposure to the attack described in the advisory that forces a fallback from v2 to v1, without having to disable -privatebroadcast or -v2transport. 1/2 🧵

@caesrcd No, that will show v1. But, it will be too late. You already connected to the peer via v1 at that point. How soon does GNOBAN disconnect the peer? You will likely have already sent the inv.

@andrew_s_toth Are you saying a malicious v2 peer would end up getting a v1 handshake, but in getpeerinfo it would still show transport_protocol_type as v2?

@caesrcd Sorry but that's not how this bug works. The addrman entry will show a v2 peer, and the downgrade does not update the entry in addrman. One of the workarounds is to disable v2 transport entirely.

@andrew_s_toth But malicious v2 peers would need to trigger a v2 handshake failure and then try again as v1. The moment they successfully complete a v1 handshake, they’ll get banned immediately.

@PortlandHODL cc @sadeeq_ismaela

Bitcoin Core v31 `estimatesmartfee` on Signet with near empty blocks reporting values of 1684 sats/vByte to get confirmed in 6 or fewer blocks. These feerates are borderline unsafe for most UTXOs and heavily overpay. After 7 blocks time it drops to .1 sats/vByte.

@caesrcd @murchandamus @L0RINC I wonder if disabling seek compaction will fix this. The constant compaction having to constantly rewrite files could be causing the churn here.

@murchandamus @L0RINC Yeah, I was surprised too. It starts swapping even with 24 GB of RAM and an 8 GB dbcache once the sync reaches block 880,000. Around block 820,000, the chainstate uses roughly 9 GB, but by block 880,000 it grows to about 13 GB.

Can someone explain this? IBD with a +10GB dbcache runs fine until ~820k blocks, then disk reads suddenly explode and I/O becomes the bottleneck. If the cache is supposed to prevent this, why does it happen right there? #Bitcoin #Benchmark #Performance

@murchandamus Congrats!

Woohoo, merged. github.com/bitcoin/bitcoi…

@caesrcd @4moonsettler Until around 800k the utxo set is still very small, and there are not as many random reads into it. See #issuecomment-3678847806" target="_blank" rel="nofollow noopener">github.com/bitcoin/bitcoi…. The vertical green line shows up around 800k. That PR would help with this as well.

@andrew_s_toth @4moonsettler Based on that theory, shouldn't the bottleneck have started around block 661,907 instead?

@caesrcd @4moonsettler Actually this corroborates the theory. With high dbcache you don't actually read anything from disk until ~800k blocks, since everything is cached. Once you fill up the cache and it has to be emptied is the first time you will read from disk.

@andrew_s_toth @4moonsettler Take a look at the disk I/O at the beginning (around block ~450,000).

@caesrcd @4moonsettler Actually, this looks like your disk is being throttled. You are getting 4-6ms reads. You need to increase storage performance to have higher throughput.

@4moonsettler @andrew_s_toth I didn't get a chance to generate the disk I/O graph, but I do have this screenshot from when I first started noticing the excessive disk usage.