Kattare Internet

… but was long enough some UPS systems drained. Then when power came back it came back in a surge and fried two UPS systems and many server power supplies. Repair and recovery took most of the day. Our generator contractor is coming out today to diagnose and hopefully repair

Update on our outage last week. The power went out in our industrial park and for the first time in 12 years our backup generator failed to start. (It is regularly exercised and just started less than a week prior) The outage was short, approx 15m, … 🧵

We are experiencing a variety of systems issues this morning. Working on bringing things back online. Current ETA for the majority of issues is Noon PST, 3 PM EST

Shifting a few racks around today, trying to retire some older racks that aren't deep enough for new equipment. Uptime discovered on one of the client machines we're moving: 14:24:23 up 3218 days, 3:17, 1 user, load average: 0.11, 0.15, 0.12 A shame to have to power it down

We are experiencing intermittent issues in Seattle again. Have reported it to our peers and are awaiting resolution.

@RegattaCentral Apologies, I missed this the other day. According to our peer in Seattle as of yesterday the routers have been patched against a BGP issue and should be fixed going forward.

@kattare Any ETA on the current outage?

Root cause for the issues in Seattle appears to be a failure in the software portion of the BGP routing engine in a pair of routers. Reboots were fixing it temporarily and yesterday engineers applied patches recommended by the router vendor that should take care of it long term.

@LiveFire717 Yes, unfortunately. We’ve been told a set of software patches have been applied to the affected routers now. According to the engineers that should take care of the issue.

@kattare is this a continuation from yesterday?

Issues in Seattle have continued to be an intermittent issue. The peer in Seattle is aware and working to resolve it. We do not have root cause from them at this time.

@robertgadach We have requested a root cause from the peer in Seattle, no report yet. Issues seem to be ongoing and intermittent.

@kattare Is there any update to this? Our site is still inaccessible.

We are tracking an outage at our peering in Seattle. Will update as soon as we have more info.

Traffic is flowing with Comcast again. No official word yet on the cause.

@JulieEvens3 So sorry, horrible timing. 😞

@kattare YIKES! We are about to teach an online workshop, and our Manual of California Vegetation database is hosted by Kattare. Any idea how long this will take to resolve?

We're tracking an issue with Comcast. No identifiable issues our side. Trying to find out more.

@JamesKoree Nothing new. 😞

@kattare Anything new?

We have been able to reach Comcast, they have confirmed the issue. Many in the region affected. Will post any updates as they come in.

@GigLibraries A large portion of our traffic routes through Comcast in Seattle, they are down for many regionally and are investigating. We do not have a root cause and unfortunately their NOC has been difficult to reach. (approx 1h wait times)

Confirmed what issue? Kattare Mail server has been down for over 4 hours. Please add details.

@cullylarson We're tracking an issue with Comcast at the moment. Nothing our side. Trying to find out more.

@kattare Our site is down (autowrecking.com). Any idea why?

Connectivity has been restored. No word yet as to the cause but we expect to be updated shortly.

We're tracking an outage at one of our providers. Have reported it. No update yet from the provider as to status or when service will be restored.