Today will
discuss end-to-end illustration of
how to use Azure Stream Analytics
and storing it into Blob Storage/Data Lake Storage Gen2. Blog 2 will focus on visualisation part.
This tutorial uses the example of real-time fraud detection
based on phone-call data. This will also work on other types of fraud
detection, such as credit card fraud or identity theft.
Scenario: Telecommunications and SIM fraud detection in real time.
A XYZ company has a huge amount of incoming calls data. They
want to detect fraudulent calls in real time and inform the user or block the
specific number.
To detect type of fraud, the company needs to examine
incoming phone records and look for specific patterns. In this case, calls made
around the same time in different countries/regions. Any phone records that
fall into this category are written to storage for subsequent analysis.
Prerequisites:
In this tutorial, you'll simulate phone-call data by using a
client app that generates sample phone call metadata. Some of the records that
the app produces look like fraudulent calls.
·
An Azure account.
·
The call-event
generator app, TelcoGenerator.zip, which can be downloaded from the Microsoft Download
Center.
Create an Azure Event
Hubs to ingest events
To analyse
a data stream, you ingest into azure using Azure event hub. It allows to ingest
millions of events per second and then store the event information.
Create a Namespace and
Event hub
You first
create an event hub namespace, then add an event hub to that namespace.
Azure
Portal à create resource à All services à Event hub in Analytics à Select Add ->
Remain
other values as defaults.
Click the new namespace, and in the namespace pane,
click Event Hub
Grant access to the event hub and get a connection string
In the event hub pane, click Shared access policies and
then click + Add.
After the policy has been deployed, click it in the list of
shared access policies
Find the box labeled CONNECTION STRING-PRIMARY KEY and
click the copy button next to the connection string
Connection string looks like this:
Endpoint=sb://infrasolution-eh-ns.servicebus.windows.net/;SharedAccessKeyName=asa-policy;SharedAccessKey=iuykGVOF3yOLRYodNDURCpBJSfoqKIirVdxf9w04ia4=;EntityPath=ev-frauddetection
Configure
and start the event generator application
Configure the TelcoGenerator app
Edit “telcodatagen.exe.config” The
<appSettings> section
will look like the following example. (For clarity, the lines are wrapped and
some characters have been removed from the authorization token.)
<appSettings>
<!-- Service Bus specific app setings for messaging connections
-->
<add key="EventHubName"
value="ev-frauddetection"/>
<add key="Microsoft.ServiceBus.ConnectionString"
value="Endpoint=sb://infrasolution-eh-ns.servicebus.windows.net/;SharedAccessKeyName=asa-policy;SharedAccessKey=iuykGVOF3yOLRYodNDURCpBJSfoqKIirVdxf9w04ia4="/>
</appSettings>
Start the application
telcodatagen.exe 1000 0.2 2
The parameters are:
·
Number of CDRs per hour.
·
SIM Card Fraud Probability: How often, as a percentage of all calls,
that the app should simulate a fraudulent call. The value 0.2 means that about
20% of the call records will look fraudulent.
·
Duration in hours. The number of hours that the app should run. You can
also stop the app any time by pressing Ctrl+C at the command line.
Create a Stream Analytics job to manage streaming data
In the Azure portal, click Create a resource > Internet of Things > Stream Analytics job.
Configure job Input & Output
In the dashboard or the All resources pane, find and select
the
asa-frauddetection-job Stream
Analytics job.
Run a test connection and Test Query
After configuring input and
output jobs, You will see below kind of output in overview section.
Check the
Size, its now increased to 260 KB
Superb
ReplyDelete