-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add implementation for extraction of env and logs from Docker containers #2
Conversation
- Used the docker-api client as a wrapper for accessing docker containers - Added functionality to extract env and logs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please add unit tests project
Refactor the code to be SOLID (mainly the S)
Understand what happen when you do SpringApplication.run(
public class AuthenticationTroubleshootingApplication { | ||
|
||
public static void main(String[] args) { | ||
String containerID = "8e53411614f2"; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please read from configurations (note that you usually want to work differently locally and in production)
@SpringBootApplication | ||
public class AuthenticationTroubleshootingApplication { | ||
|
||
public static void main(String[] args) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This class should contain only the logic that we need to run the application. not business logic
You might put here - general spring boot configurations (DI/logging etc.)
- Add properties file in order to remove hardcoded configurations - Add env and log models - Extract logic to separate Container Session file to interface with Docker
fef5354
to
998292e
Compare
@@ -0,0 +1,28 @@ | |||
package com.troubleshooting.model; | |||
|
|||
// Data object for a Conjur log |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Class documentation looks as follows:
/**
- Foo bar
*/
Java SDK contains a documentation generator that outputs HTML
private String logLevel; | ||
private String message; | ||
|
||
public String getLogLevel() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMHO any public method should be documented.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do you suggest it they be private? and if so why?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"If you like it than you should have put a ring on it"
If it's public it should be documented.
|
||
// Associate an environment variable and its value via key-value pairs | ||
public class EnvModel implements IEnvModel{ | ||
private String key=""; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
MIssing spaces.
IS there a reason for the initial value to be an empty string?
src/main/java/com/troubleshooting/model/IAuthnUserDataModel.java
Outdated
Show resolved
Hide resolved
@@ -0,0 +1,6 @@ | |||
package com.troubleshooting.model; | |||
|
|||
public interface IEnvModel { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Starting an interface name with a capital I is a C# convention.
an interface should have only methods.
@@ -0,0 +1,11 @@ | |||
package com.troubleshooting.model; | |||
|
|||
// Log interface for Conjur and DAP logs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Starting an interface name with a capital I is a C# convention.
an interface should have only methods.
src/main/java/com/troubleshooting/troubleshootingtool/ContainerDataRetrieval.java
Show resolved
Hide resolved
src/main/java/com/troubleshooting/troubleshootingtool/ContainerDataRetrieval.java
Show resolved
Hide resolved
src/main/java/com/troubleshooting/troubleshootingtool/ContainerDataRetrieval.java
Outdated
Show resolved
Hide resolved
src/main/java/com/troubleshooting/troubleshootingtool/ContainerDataRetrieval.java
Outdated
Show resolved
Hide resolved
|
||
// Retrieves the data returned from the Docker Client and packs the data in the appropriate data models | ||
@Component | ||
public class ContainerDataRetrieval implements IDataRetrieval { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As a rule of thumb if you don't plan for this class to be extended, declare it final, if they get injected create a setter for them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this a rule of thumb for all classes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Take into consideration that if it's not final it can be extended.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why is extended a class an issue?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if write a code that depends on that class, someone may provide a behaviour that do not expect. If you plan to let. others extend the class that is OK.
src/main/java/com/troubleshooting/troubleshootingtool/ContainerDataRetrieval.java
Outdated
Show resolved
Hide resolved
// Extract Logs | ||
@Override | ||
public LogsModel getData(String containerID) { | ||
List<String> loggingFrames = access.getLogs(containerID); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are certain that access.getLogs wii not return null and won't trow an exception?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If a container exists then there should be at startup logs. In terms of what to return, would I rethrow the null pointer
try {
} catch (NullPointerException e) {
throw e
}
or an empty object?
try {
} catch (NullPointerException e) {
return new LogsModel();
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Better to check for null rather than try and fall.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
check for null or throw null?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It depends on the logic. In java there are two types of exceptions:
- RuntimeException (means that something unexpected had happened that the user cannot be recovered)
- CheckedException (you must try or catch or declare that the method throws)
Usually one don't throw a RuntimeExcpetion and also not Java API exception but creates a BusinessExecption
class UserCreateException extends Exception {}
pubic void registerUser() throws UserCreateException {
try {
registerUser();
} catch (Throwable t) {
// Throwable is super class of runtime and checked exceptions
throw new UserCreateException("Failed to create user", t);
}
}
return logs; | ||
} | ||
|
||
// Extracts important parts of each Log entry |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it would be better off calling this method:
extractsImportantPartsOfEachLogEntry than a name which is not a verb (Methods should be verbs) and a single line comment above the method. Remember the best comment you can write is the one you didn't write.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
very good call
src/main/java/com/troubleshooting/troubleshootingtool/ContainerDataRetrieval.java
Outdated
Show resolved
Hide resolved
src/main/java/com/troubleshooting/troubleshootingtool/ContainerDataRetrieval.java
Outdated
Show resolved
Hide resolved
src/main/java/com/troubleshooting/troubleshootingtool/ContainerDataRetrieval.java
Outdated
Show resolved
Hide resolved
src/main/java/com/troubleshooting/troubleshootingtool/ContainerDataRetrieval.java
Outdated
Show resolved
Hide resolved
src/main/java/com/troubleshooting/troubleshootingtool/ContainerDataRetrieval.java
Outdated
Show resolved
Hide resolved
envsString = access.getEnv(containerID); | ||
envParser(envsString, env); | ||
} catch (Exception e) { | ||
e.printStackTrace(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here is where this error ends, is this the desired error handling?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe just a simple
} catch (Exception e) {
System.out.println("Unable to get environment variables from container...");
}
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add error info goes without saying. But I was referring to error handling strategy in general. this code means the error ends here, if you're aware of it and it's OK, then it's OK.
return envs; | ||
} | ||
|
||
// Extracts important parts of each Env entry |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
remove the comment and name the method:extractsImportantPartsOfEachEnvironmentEntry
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about formatAndPopulateEnvCollection
since that is what it is effectively doing? Or does that go against the S in SOLID?
What about for the logs method for extracting and populating the log model. I was thinking extractLogAndPopulateCollection
instead of extractImportantPartsOfLogEntry
but is that also break single responsibility?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if a method has And
in its name, it means it's doing more than one thing and thus violating the first rule of SOLID->SRP
parseAndPopulate should be split to:
parse()
populate()
} | ||
|
||
// Extracts important parts of each Env entry | ||
public void envParser(String envInstance, EnvModel env) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This method is public, should it be public?. And if it should be public, due to the fact that it does not validate the string or checking the array boundaries, it can easily cause a runtime exception.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It should definitely be private since no other classes call this. Updating this one now.
How does public make it easy to cause a runtime exception?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if it's public anyone can access it and get an error
|
||
@Override | ||
public void run(String... args) throws Exception { | ||
AuthnTroubleshootController start = new AuthnTroubleshootController(new AuthnUserDataModel(), new AuthnTroubleshootView()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Refrain from using abbreviations use Authentication
@@ -0,0 +1,13 @@ | |||
package com.troubleshooting.model; | |||
|
|||
public interface IAuthnUserDataModel { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Starting an interface name with a capital I is a C# convention.
Missing documentation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I want to differentiate between Interfaces and Regular classes that implement them. Would AuthenticationUserDataModelInterface
suffice?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No not in java look at all the interfaces in Java they do not have any name identifier.
Map, Collection, List, Stream etc.
- Removes abbreviations from methods and class names - Adds documentation that can be parsed - Adds addition checks for each log entry
- Update ConjurLog object to accurately represent Conjur logs - Accept only logs in the expected format - Fix ValidateService validateInput predicate - Add Hot refresh dev tools
src/main/java/com/troubleshooting/controller/AuthenticationTroubleshootController.java
Outdated
Show resolved
Hide resolved
src/main/java/com/troubleshooting/controller/AuthenticationTroubleshootController.java
Outdated
Show resolved
Hide resolved
src/main/java/com/troubleshooting/controller/AuthenticationTroubleshootController.java
Outdated
Show resolved
Hide resolved
src/main/java/com/troubleshooting/controller/AuthenticationTroubleshootController.java
Outdated
Show resolved
Hide resolved
src/main/java/com/troubleshooting/controller/AuthenticationTroubleshootController.java
Outdated
Show resolved
Hide resolved
src/main/java/com/troubleshooting/troubleshootingtool/ContainerDataRetrieval.java
Outdated
Show resolved
Hide resolved
String [] envParts = envInstance.split("=|\n"); | ||
|
||
for (int i = 0; i < envParts.length; i+=2) { | ||
if(Arrays.stream(includedEnv).anyMatch(envParts[i]::equals)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
suggested code:
String data = "HOSTNAME=253d906eab8e\n" +
"CONJUR_DATA_KEY=G2+iUjmifBzFlqXjmQT4+hq2WssOumjRkrzbadvILYg=\n" +
"PWD=/opt/conjur-server\n" +
"HOME=/root\n" +
"DATABASE_URL=postgres://postgres@database/postgres";
Properties properties = new Properties();
try {
properties.load(new StringReader(data));
} catch (IOException e) {
throw new ArgumentExcption(String.format("The input is not the expected format, %s", data));
}
List<String> includedEnv = Arrays.asList(new String[] {
"HOSTNAME",
"DATABASE_URL",
"PORT",
"CONJUR_ADMIN_PASSWORD",
"CONJUR_LOG_LEVEL",
"CONJUR_PASSWORD",
"CONJUR_ACCOUNT"
});
List<EnvironmentModel> environmentModels = properties.entrySet()
.stream().filter(e -> includedEnv.contains(e.getKey()))
.map(e -> new EnvironmentModel(e.getKey().toString(), e.getValue().toString()))
.collect(Collectors.toList());
EnvironmentsModel environmentsModel = new EnvironmentsModel();
environmentModels.addAll(environmentModels);
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the EnvironmentsModel environmentsModel = new EnvironmentsModel();
necessary here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
did you mean environmentsModel
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just as a code reference
src/main/java/com/troubleshooting/troubleshootingtool/IDataAccess.java
Outdated
Show resolved
Hide resolved
src/main/java/com/troubleshooting/troubleshootingtool/ProgramProperties.java
Outdated
Show resolved
Hide resolved
src/main/java/com/troubleshooting/troubleshootingtool/TroubleshootingToolApplication.java
Outdated
Show resolved
Hide resolved
// TODO Add query to property configuration | ||
LogContainerCmd logContainer = dockerClient.logContainerCmd(containerID).withStdOut(true).withStdErr(true); | ||
try { | ||
logContainer.exec(new LogContainerResultCallback() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure what you mean here :o
where? Instead of in the block below?
public void onNext(Frame item) {}
- Remove MVC objects for easier iteration - Remove "I" in interface names - Remove view because it is repetitive now that I have JSP files for views - Add AJAX and Jquery for rendering data without navigating to new page
src/main/java/com/troubleshooting/troubleshootingtool/ProgramProperties.java
Outdated
Show resolved
Hide resolved
|
||
spring.mvc.view.prefix: /WEB-INF/jsp/ | ||
spring.mvc.view.suffix: .jsp | ||
server.port = 4200 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add new line
final private List<ConjurLogModel> logList = new ArrayList<>(); | ||
|
||
public void add(ConjurLogModel log) { | ||
logList.add(log); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This method is public, if the argument is null, you will have a null entry [null] in your list.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added a check
public void add(ConjurLogModel log) {
if (log != null) {
logList.add(log);
}
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I need the method public to be reachable from the calling class, no?
List<String> loggingFrames = new ArrayList<String>(); | ||
|
||
loggingFrames = access.getLogs(containerID, query); | ||
loggingFrames.stream().forEach(entry -> extractImportantPartsOfLogEntry(entry)); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Replace lambda with method reference
src/main/java/com/troubleshooting/troubleshootingtool/DataRetrieval.java
Show resolved
Hide resolved
</head> | ||
|
||
<script src="http://code.jquery.com/jquery-1.7.1.min.js"></script> | ||
<script type="text/javascript"> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider having the script in a separate file.
@@ -0,0 +1,63 @@ | |||
package com.troubleshooting.controller; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Namespace is java reads the company domain name in reverse, so it should be com.cyberak.troubleshooing.controller
and if it relates only to docker maybe com.cyberak.docker.troubleshooing.controller
?
- Separate the documentations for clarity - Add TODO list to README
2796eaf
to
2ed0c40
Compare
This PR adds functionality for extracting env variables and logs from docker containers using the docker-api client.