Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pipeline release #570

Merged
merged 39 commits into from
Dec 29, 2023
Merged
Show file tree
Hide file tree
Changes from 34 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
eab9e3c
changes pipeline
Frooodle Dec 20, 2023
93f12d1
pipeline changes
Frooodle Dec 23, 2023
0345045
pipeline
Frooodle Dec 23, 2023
120b017
serial
Frooodle Dec 23, 2023
2f5d7ed
internal API plus brute force security
Frooodle Dec 24, 2023
d9b5d08
import clean
Frooodle Dec 24, 2023
690720f
test interface
Frooodle Dec 25, 2023
dd9dd72
Role stuff
Frooodle Dec 25, 2023
d235518
deps
Frooodle Dec 25, 2023
03d3235
Merge remote-tracking branch 'origin/main' into test
Frooodle Dec 25, 2023
7fb8f5e
create logs dir
Frooodle Dec 25, 2023
a286a92
cleanups
Frooodle Dec 25, 2023
b962e86
log remove
Frooodle Dec 25, 2023
6276f02
validate operations
Frooodle Dec 25, 2023
a138d5f
imports
Frooodle Dec 25, 2023
f2b701e
todos
Frooodle Dec 25, 2023
0fb0cb8
apply local
Frooodle Dec 25, 2023
05977aa
enableAlphaFunctionality
Frooodle Dec 26, 2023
6ee8e1e
auto disable in UI
Frooodle Dec 26, 2023
659af20
setup
Frooodle Dec 27, 2023
61ff024
close
Frooodle Dec 27, 2023
f5c285a
mild cleanup
Frooodle Dec 27, 2023
84aebe3
validate button color
Frooodle Dec 27, 2023
e9947da
changes
Frooodle Dec 27, 2023
78da44a
fix for #583
Frooodle Dec 27, 2023
3911be0
Add Demo user
Frooodle Dec 27, 2023
8acab77
contextPath fixes
Frooodle Dec 28, 2023
5fcb4e8
pipeline refactor beginnings
Frooodle Dec 28, 2023
829e98c
MaxRAMPercentage 75 for #540
Frooodle Dec 28, 2023
aeadc88
resolve .exe showing other entries it doesnt support
Frooodle Dec 28, 2023
a715dbb
cleanup
Frooodle Dec 29, 2023
5c6936b
Rework and cleanup
Frooodle Dec 29, 2023
168a0f0
changes
Frooodle Dec 29, 2023
610ff22
empty dir fix
Frooodle Dec 29, 2023
1b2df20
reviews
Frooodle Dec 29, 2023
3041e80
Merge branch 'main' into test
Frooodle Dec 29, 2023
a497ad8
api changes to get metrics working
Frooodle Dec 29, 2023
232a556
Merge branch 'test' of [email protected]:Frooodle/Stirling-PDF.git into …
Frooodle Dec 29, 2023
cbe4bca
add banner and remove unused class
Frooodle Dec 29, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ local.properties
.classpath
.project
version.properties
pipeline/

pipeline/watchedFolders/
pipeline/finishedFolders/
#### Stirling-PDF Files ###
customFiles/
configs/
Expand Down
6 changes: 4 additions & 2 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ ARG VERSION_TAG
# Set Environment Variables
ENV DOCKER_ENABLE_SECURITY=false \
HOME=/home/stirlingpdfuser \
VERSION_TAG=$VERSION_TAG
VERSION_TAG=$VERSION_TAG \
JAVA_TOOL_OPTIONS="$JAVA_TOOL_OPTIONS -XX:MaxRAMPercentage=75"
# PUID=1000 \
# PGID=1000 \
# UMASK=022 \
Expand All @@ -18,13 +19,14 @@ ENV DOCKER_ENABLE_SECURITY=false \
## mkdir -p $HOME && chown stirlingpdfuser:stirlingpdfgroup $HOME

# Set up necessary directories and permissions
RUN mkdir -p /scripts /usr/share/fonts/opentype/noto /usr/share/tesseract-ocr /configs /customFiles
RUN mkdir -p /scripts /usr/share/fonts/opentype/noto /usr/share/tesseract-ocr /configs /logs /customFiles /pipeline /pipeline/defaultWebUIConfigs /pipeline/watchedFolders /pipeline/finishedFolders
##&& \
## chown -R stirlingpdfuser:stirlingpdfgroup /scripts /usr/share/fonts/opentype/noto /usr/share/tesseract-ocr /configs /customFiles && \
## chown -R stirlingpdfuser:stirlingpdfgroup /usr/share/tesseract-ocr-original

# Copy necessary files
COPY ./scripts/* /scripts/
COPY ./pipeline/ /pipeline/
COPY src/main/resources/static/fonts/*.ttf /usr/share/fonts/opentype/noto/
COPY src/main/resources/static/fonts/*.otf /usr/share/fonts/opentype/noto/
COPY build/libs/*.jar app.jar
Expand Down
6 changes: 4 additions & 2 deletions Dockerfile-lite
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,8 @@ RUN apt-get update && \
# Set Environment Variables
ENV DOCKER_ENABLE_SECURITY=false \
HOME=/home/stirlingpdfuser \
VERSION_TAG=$VERSION_TAG
VERSION_TAG=$VERSION_TAG \
JAVA_TOOL_OPTIONS="$JAVA_TOOL_OPTIONS -XX:MaxRAMPercentage=75"
# PUID=1000 \
# PGID=1000 \
# UMASK=022 \
Expand All @@ -28,13 +29,14 @@ ENV DOCKER_ENABLE_SECURITY=false \
# mkdir -p $HOME && chown stirlingpdfuser:stirlingpdfgroup $HOME

# Set up necessary directories and permissions
RUN mkdir -p /scripts /usr/share/fonts/opentype/noto /configs /customFiles
RUN mkdir -p /scripts /usr/share/fonts/opentype/noto /configs /customFiles /logs /pipeline /pipeline/defaultWebUIConfigs /pipeline/watchedFolders /pipeline/finishedFolders

# chown -R stirlingpdfuser:stirlingpdfgroup /usr/share/fonts/opentype/noto /configs /customFiles

# Copy necessary files
COPY ./scripts/download-security-jar.sh /scripts/download-security-jar.sh
COPY ./scripts/init-without-ocr.sh /scripts/init-without-ocr.sh
COPY ./pipeline/ /pipeline/
COPY src/main/resources/static/fonts/*.ttf /usr/share/fonts/opentype/noto/
COPY src/main/resources/static/fonts/*.otf /usr/share/fonts/opentype/noto/
COPY build/libs/*.jar app.jar
Expand Down
9 changes: 5 additions & 4 deletions Dockerfile-ultra-lite
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ ARG VERSION_TAG
# Set Environment Variables
ENV DOCKER_ENABLE_SECURITY=false \
HOME=/home/stirlingpdfuser \
VERSION_TAG=$VERSION_TAG
VERSION_TAG=$VERSION_TAG \
JAVA_TOOL_OPTIONS="$JAVA_TOOL_OPTIONS -XX:MaxRAMPercentage=75"
# PUID=1000 \
# PGID=1000 \
# UMASK=022 \
Expand All @@ -18,12 +19,12 @@ ENV DOCKER_ENABLE_SECURITY=false \

# Set up necessary directories and permissions
#RUN mkdir -p /scripts /configs /customFiles && \
# chown -R stirlingpdfuser:stirlingpdfgroup /scripts /configs /customFiles

# chown -R stirlingpdfuser:stirlingpdfgroup /scripts /configs /customFiles /logs /pipeline /pipeline/defaultWebUIConfigs /pipeline/watchedFolders /pipeline/finishedFolders
RUN mkdir -p /scripts /usr/share/fonts/opentype/noto /configs /customFiles
COPY ./scripts/download-security-jar.sh /scripts/download-security-jar.sh
COPY ./scripts/init-without-ocr.sh /scripts/init-without-ocr.sh

COPY ./pipeline/ /pipeline/
COPY build/libs/*.jar app.jar

# Set font cache and permissions
Expand Down
12 changes: 6 additions & 6 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ plugins {
}

group = 'stirling.software'
version = '0.17.2'
version = '0.18.0'
sourceCompatibility = '17'

repositories {
Expand Down Expand Up @@ -48,7 +48,7 @@ launch4j {

errTitle="Encountered error, Do you have Java 17?"
downloadUrl="https://download.oracle.com/java/17/latest/jdk-17_windows-x64_bin.exe"
variables=["BROWSER_OPEN=true"]
variables=["BROWSER_OPEN=true", "ENDPOINTS_GROUPS_TO_REMOVE=CLI"]
jreMinVersion="17"

mutexName="Stirling-PDF"
Expand All @@ -68,17 +68,17 @@ dependencies {
implementation 'org.springframework:spring-webmvc:6.0.15'

implementation 'org.yaml:snakeyaml:2.1'
implementation 'org.springframework.boot:spring-boot-starter-web:3.1.6'
implementation 'org.springframework.boot:spring-boot-starter-thymeleaf:3.1.6'
implementation 'org.springframework.boot:spring-boot-starter-web:3.2.1'
implementation 'org.springframework.boot:spring-boot-starter-thymeleaf:3.2.1'

if (System.getenv('DOCKER_ENABLE_SECURITY') != 'false') {
implementation 'org.springframework.boot:spring-boot-starter-security:3.1.6'
implementation 'org.springframework.boot:spring-boot-starter-security:3.2.1'
implementation 'org.thymeleaf.extras:thymeleaf-extras-springsecurity5:3.1.2.RELEASE'
implementation "org.springframework.boot:spring-boot-starter-data-jpa"
implementation "com.h2database:h2"
}

testImplementation 'org.springframework.boot:spring-boot-starter-test:3.1.6'
testImplementation 'org.springframework.boot:spring-boot-starter-test:3.2.1'

// Batik
implementation 'org.apache.xmlgraphics:batik-all:1.17'
Expand Down
39 changes: 39 additions & 0 deletions pipeline/defaultWebUIConfigs/Prepare-pdfs-for-email.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
{
"name": "Prepare-pdfs-for-email",
"pipeline": [
{
"operation": "/api/v1/misc/repair",
"parameters": {}
},
{
"operation": "/api/v1/security/sanitize-pdf",
"parameters": {
"removeJavaScript": true,
"removeEmbeddedFiles": false,
"removeMetadata": false,
"removeLinks": false,
"removeFonts": false
}
},
{
"operation": "/api/v1/misc/compress-pdf",
"parameters": {
"optimizeLevel": 2,
"expectedOutputSize": ""
}
},
{
"operation": "/api/v1/general/split-by-size-or-count",
"parameters": {
"splitType": 0,
"splitValue": "15MB"
}
}
],
"_examples": {
"outputDir": "{outputFolder}/{folderName}",
"outputFileName": "{filename}-{pipelineName}-{date}-{time}"
},
"outputDir": "httpWebRequest",
"outputFileName": "{filename}"
}
33 changes: 33 additions & 0 deletions pipeline/defaultWebUIConfigs/split-rotate-auto-rename.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
{
"name": "split-rotate-auto-rename",
"pipeline": [
{
"operation": "/api/v1/general/split-pdf-by-sections",
"parameters": {
"horizontalDivisions": 2,
"verticalDivisions": 2,
"fileInput": "automated"
}
},
{
"operation": "/api/v1/general/rotate-pdf",
"parameters": {
"angle": 90,
"fileInput": "automated"
}
},
{
"operation": "/api/v1/misc/auto-rename",
"parameters": {
"useFirstTextAsFallback": false,
"fileInput": "automated"
}
}
],
"_examples": {
"outputDir": "{outputFolder}/{folderName}",
"outputFileName": "{filename}-{pipelineName}-{date}-{time}"
},
"outputDir": "{outputFolder}",
"outputFileName": "{filename}"
}
3 changes: 2 additions & 1 deletion src/main/java/stirling/software/SPDF/SPdfApplication.java
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,14 @@
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.core.env.Environment;
import org.springframework.scheduling.annotation.EnableScheduling;

import jakarta.annotation.PostConstruct;
import stirling.software.SPDF.config.ConfigInitializer;
import stirling.software.SPDF.utils.GeneralUtils;
@SpringBootApplication

//@EnableScheduling
@EnableScheduling
public class SPdfApplication {

@Autowired
Expand Down
5 changes: 5 additions & 0 deletions src/main/java/stirling/software/SPDF/config/AppConfig.java
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,11 @@ public String navBarText() {
return (defaultNavBar != null) ? defaultNavBar : "Stirling PDF";
}

@Bean(name = "enableAlphaFunctionality")
public boolean enableAlphaFunctionality() {
return applicationProperties.getSystem().getEnableAlphaFunctionality() != null ? applicationProperties.getSystem().getEnableAlphaFunctionality() : false;
}

@Bean(name = "rateLimit")
public boolean rateLimit() {
String appName = System.getProperty("rateLimit");
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,27 +2,46 @@

import java.io.IOException;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.security.authentication.BadCredentialsException;
import org.springframework.security.authentication.LockedException;
import org.springframework.security.core.AuthenticationException;
import org.springframework.security.web.authentication.SimpleUrlAuthenticationFailureHandler;
import org.springframework.stereotype.Component;

import jakarta.servlet.ServletException;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;

@Component
public class CustomAuthenticationFailureHandler extends SimpleUrlAuthenticationFailureHandler {

@Autowired
private final LoginAttemptService loginAttemptService;

@Autowired
public CustomAuthenticationFailureHandler(LoginAttemptService loginAttemptService) {
this.loginAttemptService = loginAttemptService;
}

@Override
public void onAuthenticationFailure(HttpServletRequest request, HttpServletResponse response, AuthenticationException exception)
throws IOException, ServletException {
String ip = request.getRemoteAddr();
logger.error("Failed login attempt from IP: " + ip);
if (exception.getClass().isAssignableFrom(BadCredentialsException.class)) {
setDefaultFailureUrl("/login?error=badcredentials");
} else if (exception.getClass().isAssignableFrom(LockedException.class)) {

String username = request.getParameter("username");
if(loginAttemptService.loginAttemptCheck(username)) {
setDefaultFailureUrl("/login?error=locked");

} else {
if (exception.getClass().isAssignableFrom(BadCredentialsException.class)) {
setDefaultFailureUrl("/login?error=badcredentials");
} else if (exception.getClass().isAssignableFrom(LockedException.class)) {
setDefaultFailureUrl("/login?error=locked");
}
}


super.onAuthenticationFailure(request, response, exception);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
package stirling.software.SPDF.config.security;

import java.io.IOException;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.security.core.Authentication;
import org.springframework.security.web.authentication.SavedRequestAwareAuthenticationSuccessHandler;
import org.springframework.security.web.savedrequest.SavedRequest;
import org.springframework.stereotype.Component;

import jakarta.servlet.ServletException;
import jakarta.servlet.http.HttpServletRequest;
import jakarta.servlet.http.HttpServletResponse;
import jakarta.servlet.http.HttpSession;

@Component
public class CustomAuthenticationSuccessHandler extends SavedRequestAwareAuthenticationSuccessHandler {

@Autowired
private LoginAttemptService loginAttemptService;

@Override
public void onAuthenticationSuccess(HttpServletRequest request, HttpServletResponse response, Authentication authentication) throws ServletException, IOException {
String username = request.getParameter("username");
loginAttemptService.loginSucceeded(username);


// Get the saved request
HttpSession session = request.getSession(false);
SavedRequest savedRequest = session != null ? (SavedRequest) session.getAttribute("SPRING_SECURITY_SAVED_REQUEST") : null;
if (savedRequest != null && !isStaticResource(savedRequest)) {
// Redirect to the original destination
super.onAuthenticationSuccess(request, response, authentication);
} else {
// Redirect to the root URL (considering context path)
getRedirectStrategy().sendRedirect(request, response, "/");
}

//super.onAuthenticationSuccess(request, response, authentication);
}

private boolean isStaticResource(SavedRequest savedRequest) {
String requestURI = savedRequest.getRedirectUrl();
return requestURI.startsWith("/css/")
|| requestURI.startsWith("/js/")
|| requestURI.startsWith("/images/")
|| requestURI.startsWith("/public/")
|| requestURI.startsWith("/pdfjs/")
|| requestURI.endsWith(".svg");
}

}
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
import java.util.stream.Collectors;

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.security.authentication.LockedException;
import org.springframework.security.core.GrantedAuthority;
import org.springframework.security.core.authority.SimpleGrantedAuthority;
import org.springframework.security.core.userdetails.UserDetails;
Expand All @@ -22,12 +23,18 @@ public class CustomUserDetailsService implements UserDetailsService {
@Autowired
private UserRepository userRepository;


@Autowired
private LoginAttemptService loginAttemptService;

@Override
public UserDetails loadUserByUsername(String username) throws UsernameNotFoundException {
User user = userRepository.findByUsername(username)
.orElseThrow(() -> new UsernameNotFoundException("No user found with username: " + username));

if (loginAttemptService.isBlocked(username)) {
throw new LockedException("Your account has been locked due to too many failed login attempts.");
}

return new org.springframework.security.core.userdetails.User(
user.getUsername(),
user.getPassword(),
Expand Down
Loading