Header Image
Active Incident

Updated 4 minutes ago

Incident Status

Security Breach

Components

Internal Monitoring

Locations

Salem, OR



November 2, 2026 1:47PM PST
November 2, 2026 9:47PM UTC
IDENTIFIED

Update (Nov 2): AlmaLinux's fixed kernels for CVE-2026-98432 are now in the regular updates for AlmaLinux 8 (4.18.0-553.182.1), 9 (5.14.0-687.64.1) and 10 (6.12.0-211.72.1). If your VM runs AlmaLinux, run "dnf update kernel" and reboot into the new kernel; you can then remove the interim nftables rule. For other distributions, check your vendor's security notices. On our side, the OpenStack servers move onto fixed kernels on Tuesday, November 3 (live migrations, no restart) and Thursday, November 5 (VMs restarted once), in the two maintenance notices on this page. Those reboots update our servers only, so VMs still need their own kernel update. If you need help, email support@osuosl.org.

November 1, 2026 10:20AM PST
November 1, 2026 6:20PM UTC
IDENTIFIED

A critical flaw in the Linux kernel's IPv4 TCP code (CVE-2026-98432, CVSS 9.8) lets a remote attacker with no login crash the kernel, and possibly run code, by sending crafted packets to any open TCP port. If your project runs Linux on an OpenStack VM or a server that you manage yourself, please install your distribution's fixed kernel as soon as it is available and reboot into it. Who needs to act: - Projects that manage their own OpenStack VMs or colocated servers running Linux. Every distribution is affected. - Hosts that OSL manages for you: we have already applied the interim mitigation below and will reboot them onto fixed kernels in our normal maintenance windows, announced here as usual. Fix status (Nov 1): - Debian: fixed. Debian 13 in 6.12.113-1 (DSA-6568-1), Debian 12 in 6.1.190-1 (DLA-4841-1). - AlmaLinux 8, 9 and 10: fixed kernels are in the testing repository (4.18.0-553.180.2, 5.14.0-687.61.2, 6.12.0-211.69.2) and not yet in the regular updates. - RHEL and Ubuntu: no fixed kernel yet; Red Hat rates the flaw Critical and Ubuntu has fixes in progress. For other distributions, including those built from RHEL, check your vendor's security notices. Interim mitigation: Until you can reboot onto a fixed kernel, drop incoming IPv4 TCP packets that carry IP options, which normal traffic does not use. Red Hat's bulletin gives this nftables rule (run as root; it lasts until the next reboot): nft add table inet cve_2026_98432 nft add chain inet cve_2026_98432 input '{ type filter hook input priority -310 ; policy accept ; }' nft add rule inet cve_2026_98432 input ip protocol tcp ip hdrlength gt 5 counter drop If the VM runs Docker or other containers, add the same chain and rule with "forward" in place of "input" so container ports are covered too. Remove it after rebooting onto the fixed kernel with: nft delete table inet cve_2026_98432 Red Hat's bulletin: https://access.redhat.com/security/vulnerabilities/RHSB-2026-016 If you are unsure whether you are affected or need help, email support@osuosl.org.

Scheduled Maintenance

Schedule

November 5, 2026 8:00AM - 2:00PM PST
November 5, 2026 4:00PM - 10:00PM UTC

Components

Internal Monitoring

Locations

Salem, OR

Description

On Thursday we will reboot some of the servers behind our OpenStack clouds to install fixes for two Linux kernel flaws: CVE-2026-46300 ("Fragnesia") and CVE-2026-98432, the critical TCP flaw described in our separate notice on this page. Unlike Tuesday's work, the VMs on these servers can't be moved to another server while they run, so each one will be shut down and started again once, for about 20–25 minutes. Date: Thursday, November 5, 2026 Time: 8:00 AM – 2:00 PM PST (16:00 – 22:00 UTC) Affected: - Every running VM on our aarch64 (ARM64) OpenStack cloud - About 20 VMs on our x86 OpenStack cloud We are emailing the owners and members of every affected project a list of their VMs that will restart. Not affected: - All other VMs on our x86 cloud. They move live on Tuesday, November 3, with no restart (separate notice). - VMs that are shut off stay shut off. What to do: - Make sure your services start on their own when your VM boots. - These reboots update our servers, not your VM's own kernel. Please also update and reboot your VM for CVE-2026-98432 (see our separate notice on this page). If a VM doesn't come back after its restart, please email support@osuosl.org to open a ticket so we can resolve it. Questions before then can go to the same address. We apologize for the interruption.

Schedule

December 15, 2026 9:00AM - 10:00AM PST
December 15, 2026 5:00PM - 6:00PM UTC

Components

FTP Mirrors

Locations

Chicago, IL, New York, NY, Salem, OR

Description

On Tuesday, December 15, 2026, we will permanently turn off FTP (ftp://) access to ftp.osuosl.org and the other OSU Open Source Lab mirror hostnames. The files themselves are not going anywhere: everything stays available over HTTP and HTTPS, and most of it over rsync. Date: Tuesday, December 15, 2026 Time: 9:00 AM – 10:00 AM PST (17:00 – 18:00 UTC) Expected impact: from this time on, FTP connections to the mirrors will be refused. HTTP, HTTPS and rsync keep working throughout. What you need to do: If you or your users download from our mirrors with an ftp:// URL, switch it to HTTPS before December 15. The paths are the same, so ftp://ftp.osuosl.org/pub/<project>/ becomes https://ftp.osuosl.org/pub/<project>/. If you reach us through a project name on another domain, such as ftp.<project>.org, use https://ftp.osuosl.org/ for HTTPS for now: those names don't have their own certificates yet, and we're working on adding them. If you run a mirror that syncs from us over FTP, switch to rsync://rsync.osuosl.org/<module>/ instead. Not every project we mirror has an rsync module; if the one you need is missing, email support@osuosl.org to request it. Affected: FTP on ftp.osuosl.org, ftp-osl.osuosl.org, ftp-chi.osuosl.org and ftp-nyc.osuosl.org, and on any other name that points at our mirrors, through a CNAME to ftp.osuosl.org or our IP addresses (for example ftp.<project>.org or <project>.osuosl.org). Not affected: HTTP and HTTPS at https://ftp.osuosl.org/, rsync at rsync://rsync.osuosl.org/, and every hostname above over HTTP and rsync (HTTPS on ftp.osuosl.org and <project>.osuosl.org names). The names stay; only the FTP protocol goes away. Why: FTP is an old protocol that most major projects have moved away from. Anonymous FTP has no practical way to encrypt a download or confirm that a file wasn't altered on the way to you; HTTPS does both, which is why we point you there. FTP also opens a separate connection on a negotiated port for every transfer, which makes it hard to run reliably behind today's firewalls, NAT and load balancers, and it can't be cached or accelerated the way HTTP can. For reasons like these, Chrome 88 and Firefox 90 removed FTP support in 2021, and Debian and kernel.org retired their own FTP services in 2017. Every file we mirror is already available over HTTP and HTTPS, and most of it over rsync. More background: https://osuosl.org/blog/ftp-protocol-retirement/ If you notice any issues after the change, or have questions or concerns before then, please email support@osuosl.org.

Core IPv6 Network




Operational

Core IPv4 Network




Operational

DNS




Operational

Mail Relays




Operational

FTP Mirrors




Operational

Internal Monitoring




Operational

External Services

Core Network Uptime

100.00%




Core Network Response Time

116.76ms




ftp.osuosl.org Response Time

357.04ms




Core Network Uptime

100.00%




Core Network Response Time

120.50ms




ftp.osuosl.org Response Time

331.13ms




Core Network Uptime

100.00%




Core Network Response Time

97.03ms




ftp.osuosl.org Response Time

283.58ms




Locations

History (Last 7 days)

Description

On Tuesday we will reboot the servers behind most of our x86 OpenStack cloud to install Linux kernel security updates. Before each server reboots, we move its running VMs to another server live, so they keep running: no restart and no reboot of your VM. Date: Tuesday, November 3, 2026 Time: 8:30 AM – 3:00 PM PST (16:30 – 23:00 UTC) Expected impact: none expected; a busy VM may pause for a second or two when it switches servers, and its network connections stay up What's happening: The updates fix CVE-2026-46300 ("Fragnesia"), an actively exploited Linux kernel flaw, and CVE-2026-98432, a critical flaw in the kernel's TCP code. Both are already mitigated on these servers, and the reboot loads the fixed kernel. We move about 260 VMs across 74 projects, one server at a time. Affected: - Running VMs on our x86 OpenStack cloud that use its shared storage, which is most x86 VMs Not affected: - VMs that are shut off stay shut off and are not moved. - VMs on our aarch64 cloud, x86 VMs on local disks, and three x86 VMs that can't be moved safely are restarted on Thursday, November 5 instead (separate notice). Their owners get a direct email listing them. You don't need to do anything. These reboots update our servers, not your VM's own kernel, so please still update and reboot your VM for CVE-2026-98432 (see our separate notice on this page). If a VM becomes unreachable during or after the window, please email support@osuosl.org to open a ticket so we can resolve it.


Components

Internal Monitoring


Locations

Salem, OR


Schedule

November 3, 2026 8:30AM - November 3, 2026 3:00PM PST
November 3, 2026 4:30PM - November 3, 2026 11:00PM UTC



November 3, 2026 1:51PM PST
November 3, 2026 9:51PM UTC
UPDATE

Today's OpenStack live migrations are complete, and all but one of the running VMs we moved did so without a restart. One VM stalled while moving and had to be restarted at 12:40 PM PST (20:40 UTC); it was back up within a minute, and we have contacted its owners directly. A few VMs lost network access for several minutes after their move, because their network port stayed attached to the old server until we moved it by hand. If a VM of yours is unreachable, please email support@osuosl.org. We apologize for the interruption.

November 3, 2026 8:02AM PST
November 3, 2026 4:02PM UTC
UPDATE

The OpenStack live migrations have started. Running VMs move between servers without a restart.

Incident Status

Security Breach


Components

Internal Monitoring


Locations

Salem, OR




November 1, 2026 12:58PM PST
November 1, 2026 8:58PM UTC
RESOLVED

We're resolving this incident. Every PHP host that OSL manages for you runs the fixed PHP packages from October 21, or Remi's backported fix on PHP 7.4, and session upload progress stays turned off on all of them. If you run PHP on a VM or server you manage: - Keep session.upload_progress.enabled=0 in php.ini or your PHP-FPM pool settings. PHP has not yet released a fix for CVE-2026-115560, and that setting is its only mitigation until PHP's next regular release. - When that release comes out, update to it and restart PHP-FPM or your web server. Sites still on PHP 7.4 on OSL-managed hosts keep Remi's backported fix while we move them to supported versions with each project directly, through December 2. We won't post further updates here for those moves; each project hears from us on its own ticket. If you need help, email support@osuosl.org.

October 26, 2026 10:26AM PDT
October 26, 2026 5:26PM UTC
MONITORING

Every PHP host that OSL manages for you is still running fixed PHP packages, and session upload progress is still turned off on all of them. On October 23 PHP published an advisory for a further issue, CVE-2026-115560, which has no patch yet. PHP's only mitigation for it is the same setting, session.upload_progress.enabled=0, so OSL-managed hosts need no further change. PHP will ship the fix in its next regular release. If you run PHP on a VM or server you manage: - Keep session.upload_progress.enabled=0 in php.ini or your PHP-FPM pool settings until PHP's next release. A .user.ini file or ini_set() is not enough. - If you haven't installed the October 21 fixes yet, please do so now (see our previous update). Since our last update, no OSL-managed host runs PHP 8.1 any more. Sites still on PHP 7.4 keep Remi's backported fix while we move them to supported versions with their owners: the phpBB hosts through November 18, and the sites on our shared web hosts by December 2. We'll keep this incident open until those moves are done. If you need help, email support@osuosl.org.

October 21, 2026 9:49AM PDT
October 21, 2026 4:49PM UTC
MONITORING

Every PHP host that OSL manages for you is now running fixed PHP packages, as of 9:15 AM PDT (16:15 UTC). We installed Remi's fixed builds (8.2.35, 8.3.36, 8.4.27 and 8.5.12, plus Remi's backported fix for PHP 7.4 and 8.1) and restarted PHP. The session upload progress stopgap stays in place as an extra layer. If you run PHP on a VM or server you manage, please update now: an exploit for these flaws has been published. - PHP 8.2 to 8.5: update to 8.2.35, 8.3.36, 8.4.27 or 8.5.12 from Remi or your distribution's security updates, then restart PHP-FPM or your web server. - PHP 7.4 or 8.1 from Remi's repository: update to 7.4.33-31 or 8.1.34-6, which carry Remi's backported fix, then restart. These versions are still end-of-life, so plan a move to PHP 8.3 or 8.4. - Anything else, or if you can't update yet: set session.upload_progress.enabled=0 in php.ini and restart PHP. It blocks the published attack path but is not a complete fix. Sites on PHP 7.4 and 8.1 on our shared web hosts have Remi's backported fix. We will contact site owners directly about moving those sites to a supported PHP version. We'll keep this incident open while hosted projects update. If you need help, email support@osuosl.org.

October 21, 2026 7:37AM PDT
October 21, 2026 2:37PM UTC
IDENTIFIED

PHP has released the fixes: 8.2.35, 8.3.36, 8.4.27 and 8.5.12, published at 6:34 AM PDT (13:34 UTC). Both flaws allow unauthenticated remote code execution. Sites that run PHP through Apache's mod_php are the most exposed, and PHP-FPM is affected too. If you run PHP on a VM or server you manage, update to the fixed release for your branch as soon as your package source has it, then restart PHP-FPM or your web server. For AlmaLinux and RHEL 8 or newer, Remi's repository is publishing the fixed packages today; there will be no fixed Remi packages for EL7. PHP 8.1 and older get no fix from PHP. Until you can move to 8.3 or 8.4, PHP's advisory suggests setting session.upload_progress.enabled=0 in php.ini and restarting PHP. That blocks one demonstrated attack path but is not a complete fix. Advisories: https://github.com/php/php-src/security/advisories/GHSA-w8hw-w9fx-j79w https://github.com/php/php-src/security/advisories/GHSA-6wmf-4rv5-p3rp What we are doing on the hosts OSL manages for you: - Done: we have turned session upload progress off (the stopgap above) on every running PHP host we manage. - Today: we install the fixed PHP packages as soon as Remi publishes them, and restart PHP. - Next: hosts still on PHP 7.4, 8.0 or 8.1 are moving to supported PHP versions. If your site is on one of our shared web hosts, we will contact you directly about moving it. If you need help, email support@osuosl.org.

October 20, 2026 10:22PM PDT
October 21, 2026 5:22AM UTC
IDENTIFIED

PHP has announced two critical security vulnerabilities and will publish fixed releases on Wednesday, Oct 21 at 7:00 AM PDT (14:00 UTC). If your project runs PHP on a VM or server that you manage yourself, please update PHP as soon as your package source has the fix, and restart PHP-FPM or your web server afterwards. Who needs to act: - Projects that manage their own OpenStack VMs or colocated servers and run PHP (for example WordPress, Drupal, MediaWiki, Nextcloud, phpBB or DokuWiki). - Hosts that OSL manages for you: we are patching these ourselves today, and we will contact you directly if your site needs application changes. What PHP has said: PHP rates both flaws critical (CVSS 9.2) and has not yet published which components are affected. Security researchers believe they allow remote code execution and affect PHP 5 through 8. Fixes will ship only for the supported branches 8.2, 8.3, 8.4 and 8.5. PHP 8.1 and older are end-of-life and will not get an official fix. What to do: - PHP 8.2 or newer: install your vendor's update when it appears (Remi, Ubuntu or Debian security updates, or a rebuilt Docker image), then restart PHP-FPM or the web server. - PHP 8.1 or older: plan a move to PHP 8.3 or 8.4 now. PHP 8.2's security support also ends on Dec 31, 2026. Remi's repository has backported security fixes to 7.4, 8.0 and 8.1 on a best-effort basis in the past, which can serve as a stopgap. - An end-of-life OS (CentOS 7, Ubuntu 20.04 without Ubuntu Pro, Debian 11): no free fix is expected, so please plan an OS upgrade. If you are unsure whether you are affected or need help, email support@osuosl.org.

Description

On Friday morning we will reboot a number of OSL-hosted servers to install a Linux kernel security update. Each affected service below will be down for a few minutes, once, during the window. Date: Friday, October 30, 2026 Time: 8:00 AM – 12:00 PM PDT (15:00 – 19:00 UTC) Expected impact: about 5 minutes of downtime per service while its server reboots; we reboot one server at a time What's happening: The update fixes CVE-2026-46300 ("Fragnesia"), a local privilege escalation in the Linux kernel's IPsec (ESP) code that is being actively exploited. As a temporary measure the affected kernel modules are already disabled on our hosts. The fixed kernels are installed, so a reboot is the only step left. Affected: - phpBB websites and services - Linux Foundation sites we host, including their mailing lists, archives and bug tracker - OSRF / ROS - Deluge - Snowdrift.coop, including CiviCRM and its mail - OpenMRS - Mozillazine - Buildroot's source mirror - Inkscape's Matrix server - GFortran's Mattermost - Cinc project services - A few other VMs hosted on our OpenStack cloud - Our FTP mirrors, one at a time - A POWER GPU development machine - OSL services: shell accounts and their home directories, chat, Supermarket, WebDAV and some internal web apps - Projects using OSL's shared MySQL or PostgreSQL databases may see a few seconds of failed connections while each database fails over to its standby. Not affected: - OSL's mail relays and DNS servers are rebooted one at a time with their partners still serving, so mail and name service stay up. - On Thursday, October 29 we will also reboot our Ceph storage hosts and Ganeti hypervisors one at a time. Storage stays online and Ganeti VMs are moved live to another node first, so no downtime is expected. - OpenStack hypervisors follow the week of November 2, also with live migration; running instances are not expected to see any downtime. You don't need to do anything. If a service doesn't come back after its reboot, or you notice any issues after the work is complete, please email support@osuosl.org to open a ticket so we can resolve them. Questions or concerns before then can go to the same address. We apologize for the interruption.


Components

FTP Mirrors, Internal Monitoring


Locations

Chicago, IL, New York, NY, Salem, OR


Schedule

October 30, 2026 8:00AM - October 30, 2026 12:00PM PDT
October 30, 2026 3:00PM - October 30, 2026 7:00PM UTC



October 30, 2026 12:00PM PDT
October 30, 2026 7:00PM UTC
UPDATE

Kernel security reboots are complete and all affected services are back online.

October 30, 2026 8:05AM PDT
October 30, 2026 3:05PM UTC
UPDATE

Work is under way. One change from the announcement: about 20 VMs on one of our Ganeti clusters will be restarted once, for 1–2 minutes each, instead of being moved live to another server, because the server they have to move to has an older CPU. The restart also moves them onto the fixed kernel, so they will not be rebooted a second time. These include phpBB, Snowdrift.coop, Linux Foundation, Buildroot and Mozillazine VMs already listed, plus some of the project websites on our shared web servers and a few OSL internal apps. On Thursday most Ganeti VMs moved live as planned, but a few had an unplanned restart, including about 10 minutes of downtime for the phpBB website. We're sorry about that; restarting these VMs on purpose, within today's window, is how we're avoiding it today. If a service doesn't come back after its restart, please email support@osuosl.org.

October 30, 2026 8:00AM PDT
October 30, 2026 3:00PM UTC
UPDATE

Kernel security reboots are starting. Each affected service will be down for a few minutes, one host at a time.