Project Introduction
Background
The eGaming Application Support (eGas) team at BCLC is responsible for supporting all platforms that allow users to access different entertainment products such as eCasino games, lottery, online sports, live casino, and others. These also include the maintenance and supervision of services such as login, payments, game integration, etc. Thus, to be able to ensure service and proper performance, the team utilizes an Application Performance Monitoring (APM) tool called Dynatrace that reports on the system and the services running in it to enable observability, analysis, and automation of tasks to keep the system running as expected. This assists by alerting on unusual behaviour, performance metrics, or thresholds previously set allowing the eGas team to act accordingly before a major failure occurs.
Problem
The eGas team has not been able to do a deep dive to determine what needs to be monitored closely and what other key parts of the system could be included in the reporting and alerting. Thus, allowing to classify what is valuable to the team and what isn’t, what impacts are there when certain applications or services experience issues, what thresholds are appropriate to alert on, and opportunities to implement auto-remediation. This impacts the team’s ability to investigate issues since the team currently receives alerts for expected behaviors that distract and clutter the working environment when an actual issue appears, for example, some alerts may just be due to an increase in traffic on the website which could be even considered beneficial and not an actual issue. In the same way, by not having crucial components monitored and alerting, this
makes it more difficult to pinpoint the root cause of the problem which then would keep affecting as the solution may be temporarily remediating the problem.
Solution
Conduct a deep dive into the eGaming applications and some of the underlying infrastructure supported by the eGas team using the Dynatrace APM tool. Analyze application behaviors and determine what is normal, and what abnormal behaviour would look like. Assess the current monitoring set and make recommendations on thresholds that can be configured in our alerts to reduce the number of false positives being alerted. Additionally, develop an understanding of the application functions and add tags to different components explaining their functions and areas that would be impacted if they were experiencing issues. Look for opportunities to implement synthetic monitoring to test certain functionality and availability from a user perspective and find opportunities for auto-remediation. To achieve this, collaboration with eGas, the APM team, Data Centre Operations, and likely some of the cross-functional product teams during the duration of the project is needed.
Software Development Model
This project will be completed following the agile methodology, which is based on small increments of work divided into a couple of weeks at the time to achieve each milestone. The team works on Sprints where Epics and Stories are used to compile the tasks that need to be completed based on functionality.
1. Collection of requirements per functionality. Requirements should include the main data that needs to be observed to ensure the team can be responsive when an issue appears without further escalation.
2. For each deliverable per functionality:
1. Analyze requirements.
2. Collect the most important factors to observe and need alerts.
3. Implement the solution by monitoring said services.
4. Test the implementation.
5. Review the deliverables and results with the client.
3. If successful, do final refinement to it.
4. Repeat steps. Communication with the client will be consistent throughout the duration of the project.
Justification
This project will be completed along with the APM team, which uses the agile methodology. Their main goal is to provide insights and enable processes to proactively reduce incidents, enhance application performance observability, and improve the player and the employee experience. Thus, as a member of the team, the project will also be focused on this methodology. The benefits of being part of an agile team are that there is consistent communication between team members, the team focuses on one goal at a time, the work is broken down into small actionable items to be completed within a sprint, and the ability to track the progress made. Additionally, it provides requirements clarity that allows to deliver value consistently. As the work is divided by business function, allows to deliver value per functionality completed and communicate with each stakeholder sooner rather than waiting for the whole system to be monitored and completed.
Paola Flores Aguirre – T00651883