A Clojure Library to query logs in Cloudwatch and S3

Sherlog provides APIs to

• Query Cloudwatch Log Streams
• Create Metric Filters on Cloudwatch Logs
• Create Alarms on these Metric filters

Usage

Add below git coordinates in deps.edn

omnyway-labs/sherlog
{:git/url "https://github.com/omnyway-labs/sherlog.git",
 :sha "128efe1f003c68824b2ef403c58666a4d35bd539"}

Querying Cloudwatch Logs

Make sure you have the necessary IAM permissions to Query Logs in Cloudwatch and S3

(require '[sherlog.log :as log])
(log/init! {:auth-type :profile
            :profile   (System/getenv "AWS_PROFILE")
            :region    "us-east-1"})

Other AWS auth-types:

{:auth-type :env} ;; uses the AWS SECRET environment variables
or {:auth-type :default}

(log/search log-group PATTERN DURATION-IN-SECS)

Where PATTERN is a clojure map or a jq-like Pattern supported by Cloudwatch Examples:

(log/search log-group "ERROR" 3000)
(log/search log-group "{ $.id = \"id123\" }" 3000)
;; search takes a map; This does an *and* of the entries in the map
(log/search log-group {:id "id123" :log-type "event"} 3000)

(log/tail log-group)
;; returns a log-seq

Cloudwatch Metric Filters

To list the filters:

(log/list-filters log-group)

To create a Metric Filter, you could

(log/create-filter log-group name pattern namespace initial-value)
;; example:
(log/create-filter :foo "my-metric" {:log-type "error"} "errors" 1)
;; this creates a realtime counter of errors in logs

(log/delete-filter log-group name)

Metrics from Filters

(require '[sherlog.metric :as metric])
(metric/init! {:auth-type :profile
               :profile   (System/getenv "AWS_PROFILE")
               :region    "us-east-1"})

(metric/find metric-namespace)
(metric/show "AWS/Billing" "EstimatedCharges" :currency "1d")

License - Apache 2.0

Copyright 2019 Omnyway Inc.

Licensed under the Apache License, Version 2.0 (the “License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an “AS IS” BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.