Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Integrated code lifecycle: Improve access log handling #9425

Merged
merged 23 commits into from
Oct 19, 2024
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
f47a0b2
improved access log handling
SimonEntholzer Oct 4, 2024
1c848b6
Merge branch 'develop' into chore/vcs-access-log/make-async
SimonEntholzer Oct 8, 2024
573062f
removed redundant method
SimonEntholzer Oct 8, 2024
055e470
improved naming
SimonEntholzer Oct 8, 2024
657f8dc
add docs
SimonEntholzer Oct 8, 2024
1dfedd1
added suggested changes and fixed Arch test
SimonEntholzer Oct 9, 2024
2be6283
reduced database calls
SimonEntholzer Oct 10, 2024
8c7260e
Merge branch 'develop' into chore/vcs-access-log/make-async
SimonEntholzer Oct 10, 2024
c76c05b
renamed method
SimonEntholzer Oct 10, 2024
28e3f67
remove debug action types
SimonEntholzer Oct 10, 2024
93153bf
made initial access log storing async
SimonEntholzer Oct 10, 2024
0dff694
Merge branch 'develop' into chore/vcs-access-log/make-async
SimonEntholzer Oct 13, 2024
94d8f60
Merge branch 'develop' into chore/vcs-access-log/make-async
SimonEntholzer Oct 14, 2024
7f8d970
Merge branch 'develop' into chore/vcs-access-log/make-async
SimonEntholzer Oct 14, 2024
4377396
improved shown Authentication Mechanism when clone failed
SimonEntholzer Oct 14, 2024
dd49698
remove toString
SimonEntholzer Oct 14, 2024
f85fd21
added code rabbit suggestions
SimonEntholzer Oct 14, 2024
82d1f30
added some more documentation with reference to git documentation
SimonEntholzer Oct 15, 2024
6819d5f
imporve aux repo tool tip
SimonEntholzer Oct 19, 2024
42f3dec
Merge branch 'develop' into chore/vcs-access-log/make-async
SimonEntholzer Oct 19, 2024
429baab
Revert "imporve aux repo tool tip"
SimonEntholzer Oct 19, 2024
cc3f916
revert and add
SimonEntholzer Oct 19, 2024
ecdb10e
Merge branch 'develop' into chore/vcs-access-log/make-async
SimonEntholzer Oct 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,10 @@ public void setCommitHash(String commitHash) {
this.commitHash = commitHash;
}

public void setRepositoryActionType(RepositoryActionType repositoryActionType) {
this.repositoryActionType = repositoryActionType;
}
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved

public User getUser() {
return user;
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,12 @@ default ProgrammingExerciseStudentParticipation findByExerciseIdAndStudentLoginO
return getValueElseThrow(findByExerciseIdAndStudentLogin(exerciseId, username));
}

Optional<ProgrammingExerciseStudentParticipation> findByRepositoryUri(@Param("repositoryUri") String repositoryUri);

default ProgrammingExerciseStudentParticipation findByRepositoryUriElseThrow(String repositoryUri) {
return getValueElseThrow(findByRepositoryUri(repositoryUri));
}
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved

@EntityGraph(type = LOAD, attributePaths = { "submissions" })
Optional<ProgrammingExerciseStudentParticipation> findWithSubmissionsByExerciseIdAndStudentLogin(long exerciseId, String username);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,12 @@ public interface SolutionProgrammingExerciseParticipationRepository
""")
Optional<SolutionProgrammingExerciseParticipation> findByBuildPlanIdWithResults(@Param("buildPlanId") String buildPlanId);

Optional<SolutionProgrammingExerciseParticipation> findByRepositoryUri(@Param("repositoryUri") String repositoryUri);

SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved
default SolutionProgrammingExerciseParticipation findByRepositoryUriElseThrow(String repositoryUri) {
return getValueElseThrow(findByRepositoryUri(repositoryUri));
}

SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved
@EntityGraph(type = LOAD, attributePaths = { "results", "submissions", "submissions.results" })
Optional<SolutionProgrammingExerciseParticipation> findWithEagerResultsAndSubmissionsByProgrammingExerciseId(long exerciseId);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,12 @@ default TemplateProgrammingExerciseParticipation findWithEagerResultsAndSubmissi
return getValueElseThrow(findWithEagerResultsAndSubmissionsByProgrammingExerciseId(exerciseId));
}

Optional<TemplateProgrammingExerciseParticipation> findByRepositoryUri(@Param("repositoryUri") String repositoryUri);

default TemplateProgrammingExerciseParticipation findByRepositoryUriElseThrow(String repositoryUri) {
return getValueElseThrow(findByRepositoryUri(repositoryUri));
}
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved

@EntityGraph(type = LOAD, attributePaths = { "results", "results.feedbacks", "results.feedbacks.testCase", "submissions" })
Optional<TemplateProgrammingExerciseParticipation> findWithEagerResultsAndFeedbacksAndTestCasesAndSubmissionsByProgrammingExerciseId(long exerciseId);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,14 +33,14 @@ public interface VcsAccessLogRepository extends ArtemisJpaRepository<VcsAccessLo
* @return a log entry belonging to the participationId, which has no commit hash
*/
@Query("""

SELECT vcsAccessLog
FROM VcsAccessLog vcsAccessLog
WHERE vcsAccessLog.participation.id = :participationId
AND vcsAccessLog.commitHash IS NULL
ORDER BY vcsAccessLog.timestamp DESC
LIMIT 1
""")
Optional<VcsAccessLog> findNewestByParticipationIdWhereCommitHashIsNull(@Param("participationId") long participationId);
FROM VcsAccessLog vcsAccessLog
WHERE vcsAccessLog.participation.id = :participationId
ORDER BY vcsAccessLog.timestamp DESC
LIMIT 1
""")
Optional<VcsAccessLog> findNewestByParticipationId(@Param("participationId") long participationId);
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved

/**
* Retrieves a list of {@link VcsAccessLog} entities associated with the specified participation ID.
Expand All @@ -62,7 +62,6 @@ public interface VcsAccessLogRepository extends ArtemisJpaRepository<VcsAccessLo
* The results are ordered by the log ID in ascending order.
*
* @param date The date before which all log ids should be fetched
*
* @return a list of ids of the access logs, which have a timestamp before the date
*/
@Query("""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -501,6 +501,27 @@ public ProgrammingExerciseParticipation getParticipationForRepository(Programmin
return findStudentParticipationByExerciseAndStudentLoginAndTestRunOrThrow(exercise, repositoryTypeOrUserName, isPracticeRepository, withSubmissions);
}

/**
* Get the participation for a given repository url and a repository type or user name. This method is used by the local VC system to get the
* participation for logging operations on the repository.
*
* @param repositoryTypeOrUserName the name of the user or the type of the repository
* @param repositoryURI the participation's repository URL
* @return the participation belonging to the provided repositoryURI and repository type or username
*/
public ProgrammingExerciseParticipation getParticipationForRepository(String repositoryTypeOrUserName, String repositoryURI) {
if (repositoryTypeOrUserName.equals(RepositoryType.SOLUTION.toString()) || repositoryTypeOrUserName.equals(RepositoryType.TESTS.toString())) {
return solutionParticipationRepository.findByRepositoryUriElseThrow(repositoryURI);
}
if (repositoryTypeOrUserName.equals(RepositoryType.TEMPLATE.toString())) {
return templateParticipationRepository.findByRepositoryUriElseThrow(repositoryURI);
}
if (repositoryTypeOrUserName.equals(RepositoryType.AUXILIARY.toString())) {
throw new EntityNotFoundException("Auxiliary repositories do not have participations.");
}
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved
return studentParticipationRepository.findByRepositoryUriElseThrow(repositoryURI);
}
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved

/**
* Get the commits information for the given participation.
*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

import org.eclipse.jgit.http.server.GitServlet;
import org.eclipse.jgit.transport.ReceivePack;
import org.eclipse.jgit.transport.UploadPack;
import org.springframework.context.annotation.Profile;
import org.springframework.stereotype.Service;

Expand Down Expand Up @@ -55,5 +56,13 @@ public void init() {
receivePack.setPostReceiveHook(new LocalVCPostPushHook(localVCServletService));
return receivePack;
});

this.setUploadPackFactory((request, repository) -> {
UploadPack uploadPack = new UploadPack(repository);

// Add the custom pre-upload hook
uploadPack.setPreUploadHook(new LocalVCFetchPreUploadHook(localVCServletService, request));
return uploadPack;
});
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.security.core.AuthenticationException;
import org.springframework.web.filter.OncePerRequestFilter;

import de.tum.cit.aet.artemis.core.exception.localvc.LocalVCAuthException;
Expand Down Expand Up @@ -44,6 +45,11 @@ public void doFilterInternal(HttpServletRequest servletRequest, HttpServletRespo
servletResponse.setStatus(localVCServletService.getHttpStatusForException(e, servletRequest.getRequestURI()));
return;
}
catch (AuthenticationException e) {
// intercept failed authentication to log it in the VCS access log
localVCServletService.createVCSAccessLogForFailedAuthenticationAttempt(servletRequest);
throw e;
}

filterChain.doFilter(servletRequest, servletResponse);
}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
package de.tum.cit.aet.artemis.programming.service.localvc;

import java.util.Collection;

import jakarta.servlet.http.HttpServletRequest;

import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.transport.PreUploadHook;
import org.eclipse.jgit.transport.UploadPack;

public class LocalVCFetchPreUploadHook implements PreUploadHook {

private final LocalVCServletService localVCServletService;

private final HttpServletRequest request;

public LocalVCFetchPreUploadHook(LocalVCServletService localVCServletService, HttpServletRequest request) {
this.localVCServletService = localVCServletService;
this.request = request;
}

@Override
public void onBeginNegotiateRound(UploadPack uploadPack, Collection<? extends ObjectId> collection, int clientOffered) {
localVCServletService.updateVCSAccessLogForCloneAndPullHTTPS(request, clientOffered);
}

@Override
public void onEndNegotiateRound(UploadPack uploadPack, Collection<? extends ObjectId> collection, int i, int i1, boolean b) {
}

@Override
public void onSendPack(UploadPack uploadPack, Collection<? extends ObjectId> collection, Collection<? extends ObjectId> collection1) {
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
package de.tum.cit.aet.artemis.programming.service.localvc;

import java.nio.file.Path;
import java.util.Collection;

import org.apache.sshd.server.session.ServerSession;
import org.eclipse.jgit.lib.ObjectId;
import org.eclipse.jgit.transport.PreUploadHook;
import org.eclipse.jgit.transport.UploadPack;

public class LocalVCFetchPreUploadHookSSH implements PreUploadHook {
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved

private final LocalVCServletService localVCServletService;

private final ServerSession serverSession;

private final Path rootDir;

public LocalVCFetchPreUploadHookSSH(LocalVCServletService localVCServletService, ServerSession serverSession, Path rootDir) {
this.localVCServletService = localVCServletService;
this.serverSession = serverSession;
this.rootDir = rootDir;
}
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved

@Override
public void onBeginNegotiateRound(UploadPack uploadPack, Collection<? extends ObjectId> collection, int clientOffered) {
localVCServletService.updateVCSAccessLogForCloneAndPullSSH(serverSession, rootDir, clientOffered);
}
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved

@Override
public void onEndNegotiateRound(UploadPack uploadPack, Collection<? extends ObjectId> collection, int i, int i1, boolean b) {
}

@Override
public void onSendPack(UploadPack uploadPack, Collection<? extends ObjectId> collection, Collection<? extends ObjectId> collection1) {
}
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved
}
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ public void doFilterInternal(HttpServletRequest servletRequest, HttpServletRespo
servletResponse.setStatus(localVCServletService.getHttpStatusForException(e, servletRequest.getRequestURI()));
return;
}
this.localVCServletService.updateVCSAccessLogForPushHTTPS(servletRequest);
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved

filterChain.doFilter(servletRequest, servletResponse);

Expand Down
Loading
Loading