Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add structured logging and Prometheus events #1767

Merged
merged 9 commits into from
Nov 20, 2019

Conversation

nambrot
Copy link
Contributor

@nambrot nambrot commented Nov 19, 2019

Description

This PR adds structured logging and Prometheus events to the attestation service.

Tested

  • Locally against namoffchainreveal

Related issues

@nambrot nambrot assigned nambrot, mcortesi and jmrossy and unassigned nambrot Nov 19, 2019
Copy link
Contributor

@mcortesi mcortesi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left some comments

@@ -1,38 +1,46 @@
import * as dotenv from 'dotenv'

// We need to load the config before some of the imports for e.g. the logger
if (process.env.CONFIG) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this doesn't sound right... imports by ES6 definition must be the first thing on a file, you can't add imports after code

import { initializeDB, initializeKit } from './db'
import { createValidatedHandler } from './request'
import { logger } from './logger'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe rename to "rootLogger"

})

// Somehow it wouldn't let me export logger without export Logger
export { Logger }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

try setting the type for const logger explicitely

const logger: Logger

import express from 'express'
import { isLeft } from 'fp-ts/lib/Either'
import * as t from 'io-ts'
import { logger } from './logger'

export enum ErrorMessages {
UNKNOWN_ERROR = 'Something went wrong',
}

export function catchAsyncErrorHandler<T>(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think asyncHandler() is good enough name :P

@@ -64,3 +68,22 @@ function serializeErrors(errors: t.Errors) {
export function respondWithError(res: express.Response, statusCode: number, error: string) {
res.status(statusCode).json({ success: false, error })
}

export type Response = Omit<express.Response, 'locals'> & {
locals: { logger: Logger } & any
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why the & any?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't entirely sure if the compiler would be capable to intersect both locals, but let me try

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Turns out

export type Response = Omit<express.Response, 'locals'> & {
  locals: { logger: Logger } & Omit<any, 'logger'>
}

was the way to go

respondWithError(res, 500, ErrorMessages.UNKNOWN_ERROR)
}
return (req: express.Request, res: Response) => {
handler(req, res)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

word of caution.... here you don't catch an edge case, suppose the handler() do some things and then return a promise... for example:

function evilHandler() {
  if (true) {
    throw new Error()
  }
  return callSomethingThatRetunsAPromise()
}

In this case... there's a throw before, you are not actually catching the error.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great catch, definitely wasn't aware

@@ -31,6 +31,8 @@
"@celo/utils": "^0.1.0",
"bignumber.js": "^7.2.0",
"body-parser": "1.19.0",
"bunyan": "1.8.12",
"bunyan-gke-stackdriver": "0.1.2",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this couple the service tightly with GCloud? Validators may want to host this on AWS/Azure

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bunyan-gke-stackdriver by @mcortesi is a plugin to bunyan to create an alternatively formatted output stream, it does not require any GCP specific functionality to build/run, and can be configured with LOG_FORMAT=stackdriver, so is opt-in

@codecov
Copy link

codecov bot commented Nov 19, 2019

Codecov Report

Merging #1767 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff            @@
##           master    #1767    +/-   ##
========================================
  Coverage   74.21%   74.21%            
========================================
  Files         278      278            
  Lines        7652     7652            
  Branches      673      957   +284     
========================================
  Hits         5679     5679            
  Misses       1856     1856            
  Partials      117      117
Flag Coverage Δ
#mobile 74.21% <ø> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 59ed052...e5ef328. Read the comment docs.

@nambrot nambrot added the automerge Have PR merge automatically when checks pass label Nov 19, 2019
@nambrot
Copy link
Contributor Author

nambrot commented Nov 20, 2019

Tests pass, but enabled SSH

@nambrot nambrot merged commit 9c8f015 into master Nov 20, 2019
@mcortesi mcortesi deleted the nambrot/attestation-service-logging-monitoring branch December 4, 2019 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automerge Have PR merge automatically when checks pass
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Validators should be able to operate the attestation service with proper logging/monitoring
3 participants