SoftBank Payment Service: Powering the Search for Payment Service Monitoring and Fraud Detection

SoftBank’s online payment system previously sent failure notifications to technical staff responsible for Softbank systems via email or Slack. This meant that SoftBank’s IT support team was responsible for reporting failures to the relevant sales department and, in turn, the sales department would report incident details to the member store where the payment failure had occurred. The IT team was solely responsible for delivering information about failures, tracing the cause of the failure, fixing it, and making the client aware of the situation. This system of failure reporting was not only time consuming, but also prevented delivery of immediate and complete oversight of relevant failure data to the sales and operations departments at SoftBank Payment Services.

Given the lack of visibility into failures when they occured, SoftBank sought a monitoring and visualization tool.They found Kibana, with its clear and easy-to-use dashboards, as the ideal product. Using Logstash to regularly update transactional data within Elasticsearch, SoftBank could then visualize transaction data immediately using KIbana and share that information across their business.

Softbank operational staff created dashboards that show successful payments in green and failed payments in red. Operators monitoring these dashboards can quickly catch changes in the volume of successful and failed payments, organized by the different payment methods, and act on them swiftly.

There are two simple rules that now let operators understand what they are seeing more intuitively, and to quickly pinpoint payment problems:

1. Watch for rapid decreases in successful, green-colored payments, and
2. Watch for rapid increases in failed, red-colored payments.

If any changes are noted, operators can quickly drill down on transactions at both the payment method level and the member store level and discover the origin.

The Elastic Stack also provides credit processing status information. Kibana dashboards can show operators information on which member stores send requests, and what error codes occur. These error codes are used to identify when incorrect credit card numbers are used, if a card has an invalid expiration date, or if the credit card is blocked or unusable. Pie charts on the dashboard provide detailed data on the number of users who are experiencing errors, as well as the credit card numbers involved. This, in turn, helps operators to determine when fraudulent credit card transactions are occurring.

Kibana dashboards can also be securely shared with other teams, both at and away from work. For example, an engineer on-call but not in the office can set up system-trigger alerts for his smartphone. Softbank uses Jenkins to capture dashboard screenshots when errors occur and, using Selenium, repeats this process every five minutes. Those screenshots are then posted to a specific Slack channel that alerts the engineer on duty.

In July of 2017, SoftBank Payment Service started using machine learning on a trial basis. As a test, Elastic machine learning was used to process data specific to daily total payment transaction trends over the course of three cycles. Any change significantly different from the forecast was identified, processed, and considered to be an anomaly. As an example, credit facilities for online services, such as smartphone games, are usually made available on the first day of every month, usually resulting in increased payments. At first, this situation was detected as an anomaly. But after three months, this pattern was recognized by machine learning, and the system stopped detecting it as a strange fluctuation in sales. Machine learning gave results precisely as expected.

To identify other potential problems, new machine learning jobs were created with different criteria, such as the total number of successful and failed payments divided into different payment methods. Member stores, product names, and error codes, including the error messages, were set as influencing factors. These factors help to identify the cause when certain anomalies are detected, for example, detecting spikes in alerts for incorrect card numbers or expiration dates. Historically, when member stores were analyzed all together, alert data could get lost among the incredible volume of transactions, making it difficult to search for problems. Now, however, when SoftBank Payment Services personnel filter for specific member stores, the Kibana dashboard lets operators immediately find fraudulent uses of credit cards. SoftBank Payment Services has realized, therefore, that visualizing the data alone is not enough when trying to identify credit card fraud.

Upon detection of an anomaly, the job name and level of severity are posted to Slack. This process also comes with a unique measurement achieved by a Slack bot which captures and adds screenshots for the machine learning dashboard.

By increasing the level of complicated jobs, personnel can detect fraud earlier and will therefore be able to enhance services for member stores. SoftBank Payment Services IT support aims to also replace the old alert system, which is based on member store thresholds or on payment methods. They are thinking of instead, adopting an anomaly-detection process based on machine learning.

SoftBank Payment Services has since expanded the test use of the Elastic Stack for data visualization to new departments. In this expansion, the IT support team looked at Excel-based sales data previously managed by the sales department. Targets, such as the number of contracts and the web traffic to service sites, were then set to track against.

For contracts, trends for sales volumes over the course of two years were converted into stacked bar charts and broken down by department or project. Heat maps let SoftBank Payment Services list targets for the sales department that could be further refined to the sales representative level or filtered by month. Any targets met were shown in blue; missed targets were in red. A map was also used to chart the annual status for member stores, at the prefecture level.

For service site traffic, the IT support team at SoftBank Payment Services was able to guess which companies were coming to their site based on the source IP addresses. Access by both contracted client companies and non-contracted companies were ranked to provide the sales department with usable information.

A dashboard was initially created by IT support, upon request from the sales department. At some point, however, the sales department started working on new tasks ranging from adding and visualizing data, to creating new dashboards. The most difficult problem of adding Excel-based data was finally solved by developing a standalone drag-and-drop tool.

These early efforts paved the way for SoftBank Payment Services to conduct analyses in greater detail, using regular expression tools and simple mathematical operations available from Kibana. At the same time, those efforts let SoftBank Payment Services handle massive amounts of data that could not otherwise be processed with Excel. These enhancements now enable anyone to enter data and create dashboards, making improvements to reporting a real collaborative effort.