Results: Model Performances in GPML

Example 1: Number of Edges and Density

Common parameters

• Number of vertices (accounts): 27,770
• Alert patterns: 100 triangles as fraud patterns and 1000 fan-in/out patterns with 5 vertices as false alerts

GPML Results

• Sparse (56,021 transactions)

• Medium (96,463 transactions)

• Dense (160,970 transactions)

• 239,385 transactions

• 276,359 transactions

Findings

In dense transaction networks, the combinations which yield the largest precision is undirected cycle feature, but the recall drops to around 0.2. The best number of edges is around 160K.

Example 2: Number of Fraud Patterns and False Alerts

Common parameters:

• Number of vertices (accounts): 27,770
• Number of edges (transactions): 160K
• Fraud pattern: Triangle (3-length cycle)
• False alert pattern: Fan-in/out with 5 accounts

GPML Results

• 50 fraud patterns (500 false alerts)

• 100 fraud patterns (1,000 false alerts)

• 200 fraud patterns (2,000 false alerts)

Findings

The recall values are more stable with more fraud patterns (more than 0.8 with 200 fraud patterns). The graph feature combinations of undirected cycles and egonet gain the precision and F1-score.

Example 3: Size of Fraud patterns (length of cycle patterns)

Common parameters:

• Number of vertices (accounts): 27,770
• Number of edges (transactions): 160K

100 fraud patterns and 1,000 false alerts

• Triangle (3-length cycles)

• Square (4-length cycles)

• Pentagon (5-length cycles)

200 fraud patterns and 2,000 false alerts

• Triangle

• Square

• Pentagon

Findings

The recall values are stable with the triangle fraud patterns. With the square and pentagon fraud patterns, the recall values drop to 0.4-0.5 for some features.

Example 4: Type of Fraud patterns

Common parameters:

• Number of vertices (accounts): 27,770
• Number of edges (transactions): 160K
• False alert patterns: 1000 fan-in/out patterns with 3 vertices

GPML Results

• 100 Pentagons (5-length cycles)

• 100 Fan-in/out (5 vertices)

• 200 Pentagons (5-length cycles)

• 200 Fan-in/out (5 vertices)

Findings

In fan-in/out fraud patterns, there are trade-offs between precision and recall values, and recall values dropped to 0.2-0.4 in some feature combinations.