Integration Guides
Microsoft Sentinel
Microsoft Sentinel can accept arbitrary json data as Custom Log sources, so you can ingest almost any type of Incydr data for querying within Sentinel, including Alerts, File Events, and Audit Log events.
Sentinel has two methods of ingesting JSON data as Custom Log sources: the data collector HTTP API, and the Log Analytics Linux agent. While the examples below will focus on Incydr Alerts, the patterns can be re-used for almost any type of data you can pull from the Code42 API or CLI.
Send Incydr Alerts to Sentinel with py42 and the Azure Monitor data collector API
This guide outlines how to send Incydr Alerts to Sentinel with a Python script using py42 and the data collector API.
Expand guide
Prepare your environment:
- Install py42 into your Python environment
- Create a Code42 API Client to authenticate py42. The
Alerts Read
permission is required to query alerts. - Gather your Azure Log Analytics
Workspace ID
and eitherPrimary Key
orSecondary Key
to authenticate requests pushing data into Sentinel.
Writing the script:
First, import the py42 SDK client and required classes for building your Alert query:
import py42.sdk
from py42.sdk.queries.alerts.alert_query import AlertQuery
from py42.sdk.queries.alerts.filters import DateObserved
Then construct your query. This example will search for Alerts created within the past 3 days. See py42 documentation for details on how to customize your own query.
from datetime import datetime, timedelta
date_filter = DateObserved.on_or_after(datetime.utcnow() - timedelta(days=3))
alert_query = AlertQuery(date_filter)
Initialize the py42 SDK object with your API client details:
sdk = py42.sdk.from_api_client(
host_address="https://console.us.code42.com",
client_id="<your_client_id>",
client_secret="<your_secret>"
)
Fetch the alerts with your query:
response = sdk.alerts.search(query)
The Sentinel HTTP endpoint accepts a list of JSON objects in the request body, to prepare the py42 response for sending
to Sentinel, you just need to extract the alerts
data from the py42 response and convert it to a JSON string:
import json
response = sdk.alerts.search(query)
body = json.dumps(response.data["alerts"])
Then we can use the Python example from the Data Collector API documentation to authenticate and send the Incydr Alerts to Sentinel. The example is recreated here with the py42 code replacing the example data:
import json
import requests
import datetime
import hashlib
import hmac
import base64
import py42.sdk
from py42.sdk.queries.alerts.alert_query import AlertQuery
from py42.sdk.queries.alerts.filters import DateObserved
# Update the customer ID to your Log Analytics workspace ID
customer_id = 'xxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'
# For the shared key, use either the primary or the secondary Connected Sources client authentication key
shared_key = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
# This creates a new Custom Log type for Incydr Alerts
log_type = 'IncydrAlert'
#####################
##Incydr Alert Query#
#####################
date_filter = DateObserved.on_or_after(datetime.datetime.utcnow() - datetime.timedelta(days=3))
alert_query = AlertQuery(date_filter)
sdk = py42.sdk.from_api_client(
host_address="https://console.us.code42.com",
client_id="<your_client_id>",
client_secret="<your_secret>"
)
response = sdk.alerts.search(query)
body = json.dumps(response.data["alerts"])
#####################
##Sentinel Functions#
#####################
# Build the API signature
def build_signature(customer_id, shared_key, date, content_length, method, content_type, resource):
x_headers = 'x-ms-date:' + date
string_to_hash = method + "\n" + str(content_length) + "\n" + content_type + "\n" + x_headers + "\n" + resource
bytes_to_hash = bytes(string_to_hash, encoding="utf-8")
decoded_key = base64.b64decode(shared_key)
encoded_hash = base64.b64encode(hmac.new(decoded_key, bytes_to_hash, digestmod=hashlib.sha256).digest()).decode()
authorization = "SharedKey {}:{}".format(customer_id,encoded_hash)
return authorization
# Build and send a request to the POST API
def post_data(customer_id, shared_key, body, log_type):
method = 'POST'
content_type = 'application/json'
resource = '/api/logs'
rfc1123date = datetime.datetime.utcnow().strftime('%a, %d %b %Y %H:%M:%S GMT')
content_length = len(body)
signature = build_signature(customer_id, shared_key, rfc1123date, content_length, method, content_type, resource)
uri = 'https://' + customer_id + '.ods.opinsights.azure.com' + resource + '?api-version=2016-04-01'
headers = {
'content-type': content_type,
'Authorization': signature,
'Log-Type': log_type,
'x-ms-date': rfc1123date
}
response = requests.post(uri,data=body, headers=headers)
if (response.status_code >= 200 and response.status_code <= 299):
print('Accepted')
else:
print("Response code: {}".format(response.status_code))
post_data(customer_id, shared_key, body, log_type)
After the script is run, you should be able to query the IncydrAlert_CL
log table and see the Incydr Alert data.
Send Incydr Alerts to Sentinel with the Code42 CLI and the Azure Log Analytics Linux agent
This guide outlines how to configure a custom JSON source for ingest using the Azure Log Analytics agent The Code42 CLI can then be installed on the same machine to fetch alerts periodically using the Log Analytics agent configuration.
Expand guide
Prepare your environment:
- Install the Azure Log Analytics Agent
- Install the Code42 CLI
- Configure a Code42 CLI profile for the
omsagent
user:- Become the
omsagent
user:sudo su omsagent
- Create a new Code42 CLI profile:
code42 profile create-api-client --name sentinel --server <code42_console_domain> --api-client-id <your_id> --secret <your_secret>
- Test the command you'll be using to ingest Inycdr data, for example:
code42 alerts search --begin 3d --format raw-json
- Become the
Configure the Log Analytics Agent (omsagent
)
Following the steps from Azure Documentation,
below is an example config that ingests Incydr Alerts using the Code42 CLI. The configuration examples can be added to
/etc/opt/microsoft/omsagent/<workspace id>/conf/omsagent.conf
or in their own separate file in the
/etc/opt/microsoft/omsagent/<workspace id>/conf/omsagent.d/
folder.
The input configuration
<source>
type exec
command code42 alerts search -b 1d --format raw-json --checkpoint sentinel
format json
tag oms.api.incydr
run_interval 1h
</source>
The code42
command has two important options, the --format raw-json
output causes the CLI to write each Alert JSON
object to stdout on its own line, and the --checkpoint sentinel
option tells the CLI to store the last retrieved alert
timestamp in the CLI profile, and subsequent runs will use that checkpoint date as the starting point for the next query.
This prevents duplicate alerts from being ingested, as the query will only search for alerts after the last seen one.
The tag
config value defines what Custom Log table the events will be written to. Whatever is after oms.api.
in the
tag will become a new Custom Log table (in this example the table name becomes incydr_CL
).
The output configuration
<match oms.api.incydr>
type out_oms_api
log_level info
buffer_chunk_limit 5m
buffer_type file
buffer_path /var/opt/microsoft/omsagent/<workspace id>/state/out_oms_api_incydr*.buffer
buffer_queue_limit 10
flush_interval 20s
retry_limit 10
retry_wait 30s
</match>
Once the configuration is saved, restart the omsagent
by running: /opt/microsoft/omsagent/bin/service_control restart
.
You should start seeing Incydr alerts being populated in the incydr_CL
table shortly.
For issues troubleshooting ingest using the Log Analytics agent, see Microsoft's Troubleshooting FAQ.