Ticket 20230418_1
| Ticket Number: | 20230418_1 | Ticket State: | CLOSED | ||
|---|---|---|---|---|---|
| Ticket Opened: | 2023-04-18 11:42 | Ticket Closed: | 2023-04-20 16:50 | ||
| Ticket Description: | Scheduled emergency maintenance optical card in Lausanne (19.-20.4.2023) | ||||
| Scheduled Starttime: | 2023-04-19 06:00:00 | Scheduled Endtime: | 2023-04-20 15:00:00 | ||
Problem Description:
Our system vendor needs to investigate a misbehaving optical card in Lausanne to prevent a sudden failure later. To minimize impact on customer traffic we will switch links to their backup paths ahead of time early in the morning when traffic is expected to still be low. This will generate a short flap, therefore this impact is marked as a partial loss of connectivity.The vendor will then be able to investigate and we will move traffic back to the original path later. The timing of this will depend on the vendors findings. We will inform you about this ahead of time.
Redundancy is only lost for around an hour during the vendors work tomorrow morning. The system can still perform through the affected card at the moment.
Affected:
From 2023-04-19 13:00 until 2023-04-20 15:00Impact: no more redundancy
Service: P2P/OPN
Sites: BELWUE-KIT-CE, EXOSCALE-GEANT, L2VPN-RKAU-INDIANAPOLIS, L2VPN-RKAU-SANTACLARA, SYNGENTA-INTERNET2, SYNGENTA-JANET
From 2023-04-19 08:30 until 2023-04-20 10:30
Impact: no more redundancy
Service: P2P/OPN
Sites: BELWUE-KIT-CE, EXOSCALE-GEANT, L2VPN-RKAU-INDIANAPOLIS, L2VPN-RKAU-SANTACLARA, SYNGENTA-INTERNET2, SYNGENTA-JANET
From 2023-04-19 06:00 until 2023-04-19 07:00
Impact: Partial loss of connectivity
Service: P2P/OPN
Sites: BELWUE-KIT-CE, EXOSCALE-GEANT, L2VPN-RKAU-INDIANAPOLIS, L2VPN-RKAU-SANTACLARA, SYNGENTA-INTERNET2, SYNGENTA-JANET
Actions:
2023-04-20 16:50
The card was replaced between 10:53 and 11:37 and redundancy is reestablished since then. The optical links are not yet switched back to their primary paths, this will be done separately later.
2023-04-19 13:05
Our vendor has investigated the card, and attempted a remote fix of the WSS function, but the fix did not succeed. Additionally we noticed that during the maintenance last night, announced in ticket 20230413_1, the optical card did not perform as expected, indicating it has a deeper issue. It will be replaced tomorrow. For now the optical links remain on the alternative paths, and are expected to be stable, but without redundancy. We are extending the maintenance window to tomorrow to include the time needed to replace the card.
2023-04-19 06:24
Optical paths switched to backup where necessary. No further flaps expected for the rest of the morning.
All times are in local Swiss time CET / CEST.
For all questions about this ticket, please send mail to noc@switch.ch
or call +41 44 268 15 30.
