AllSecure System - Open Platform Performance Alert – Incident details

EXCHANGE PLATFORM under maintenance

Open Platform Performance Alert

Resolved
Operational
Started 4 months agoLasted about 11 hours

Affected

OPEN PLATFORM

Degraded performance from 4:00 AM to 2:57 PM

Updates
  • Resolved
    Resolved

    After monitoring the transactions from our side, we notice that the transactions are properly processed and that the problem has been solved.

  • Monitoring
    Monitoring

    We notice that the system has started to stabilize since 15:00h CEST. Therefore, we on the our system see successfully processed transactions. We certainly continue to monitor the flow of transactions.

  • Investigating
    Update

    We apologise for the continued interruption to your service and would like to assure you that we are doing everything we can to restore services to normal without further delay. The issue, as mentioned earlier appears to lie with an issue upstream with a 3rd Party Tier 1 ISP, and our own ISP providers are working with the vendor to troubleshoot the issue. The issue seems to primarily affect Mastercard traffic but we do also see a small percentage of affected VISA traffic. From a shopper perspective, 3DS verifications are the main cause of failures as the 3DS check does not get completed and times out. A high percentage of shoppers retry their 3DS check several times, eventually leading to success.

  • Investigating
    Update

    Ours attempts to move traffic to the alternate data center have not solved the issue. The issue is visible from both our datacenters using different network providers. We have heard back from our ISP providers and they reported there is a suspected issue with Tier 1 ISP that is being utilized during the transaction flow. Ours ISP providers are reaching out to gather further information on that issue and ascertain whether that could be the route cause of this issue.

  • Investigating
    Update

    As we await insights from the ISP and the acquirer, our strategy involves a methodical assessment of impact and migration of the affected VPNs from the Primary environment to the alternate site. This move is aimed at evaluating the possibility of mitigating the impact through this transition.

  • Investigating
    Update

    Our technical teams are actively working to resolve the issue, with engineers meticulously analyzing VPN logs to pinpoint the underlying cause. Concurrently, we are conducting a thorough internal review, and our ISP partners are also examining potential sources of the problem from their end.

  • Investigating
    Investigating

    We're currently experiencing a technical issue that's impacting our ability to process a subset of transactions through the Open Payments Gateway. Our initial investigation suggests there may be a problem with our internet service provider. We're in close communication with them to resolve the issue as quickly as possible.