Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix unmodified pomfile #379

Closed

Conversation

imonteroperez
Copy link
Contributor

Highlights

  • This PR tries to fix a weird behavior detected on testing junit plugin
  • If we launch the PCT CLI with a war file with some dependencies to update (for example: boostrap5-api - which is also a dependency of junit plugin) the PCT executes the testing without modifying its pomfile which seems wrong
  • This is happening if we do not provide any excludeHooks and if we provide a excludeHooks pointing to TransformPom
  • This behavior seems wrong and unrelated to the need of executing the PluginCompatTester#addSplitPluginDependencies method. Introduced in: 23e2bcb#diff-56ea8384acb1f763e2f462f02781e29ee845b4901b11f30bef2bd83ee984d51eR565
  • Once applied the change proposed in the PR, we obtain the expected modification in the pomfile:
Adding/replacing plugin dependencies for compatibility: { ... bootstrap5-api=5.2.0-1, ...}
  • Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
  • Ensure that the pull request title represents the desired changelog entry
  • Please describe what you did
  • Link to relevant issues in GitHub or Jira
  • Link to relevant pull requests, esp. upstream and downstream changes
  • Ensure you have provided tests - that demonstrates feature works or fixes the issue

@jglick
Copy link
Member

jglick commented Sep 21, 2022

…and if we provide a excludeHooks pointing to TransformPom

I.e., via #372, to take advantage of jenkinsci/maven-hpi-plugin#336. This is proposing to revert a change made in #372 for what seems like a good reason: we do not wish to ever again modify pom.xml files in the PCT checkout. I do not recall exactly how

private void addSplitPluginDependencies(String thisPlugin, MavenRunner.Config mconfig, File pluginCheckoutDir, MavenPom pom, Map<String, Plugin> otherPlugins, Map<String, String> pluginGroupIds, String coreVersion, List<PCTPlugin> overridenPlugins, String parentFolder) throws Exception {
File tmp = File.createTempFile("dependencies", ".log");
VersionNumber coreDep = null;
Map<String,VersionNumber> pluginDeps = new HashMap<>();
Map<String,VersionNumber> pluginDepsTest = new HashMap<>();
try {
if (StringUtils.isBlank(parentFolder)) {
runner.run(mconfig, pluginCheckoutDir, tmp, "dependency:resolve");
} else {
String mavenModule = getMavenModule(thisPlugin, pluginCheckoutDir, runner, mconfig);
if (StringUtils.isBlank(mavenModule)) {
throw new IOException(String.format("Unable to retrieve the Maven module for plugin %s on %s", thisPlugin, pluginCheckoutDir));
}
runner.run(mconfig, pluginCheckoutDir.getParentFile(), tmp, "dependency:resolve", "-am", "-pl", mavenModule);
}
try (BufferedReader br =
Files.newBufferedReader(tmp.toPath(), Charset.defaultCharset())) {
Pattern p = Pattern.compile("\\[INFO\\]([^:]+):([^:]+):([a-z-]+):(([^:]+):)?([^:]+):(provided|compile|runtime|system)(\\(optional\\))?.*");
Pattern p2 = Pattern.compile("\\[INFO\\]([^:]+):([^:]+):([a-z-]+):(([^:]+):)?([^:]+):(test).*");
String line;
while ((line = br.readLine()) != null) {
line = line.replace(" ", "");
Matcher m = p.matcher(line);
Matcher m2 = p2.matcher(line);
String groupId;
String artifactId;
VersionNumber version;
if (!m.matches() && !m2.matches()) {
continue;
} else if (m.matches()) {
groupId = m.group(1);
artifactId = m.group(2);
try {
version = new VersionNumber(m.group(6));
} catch (IllegalArgumentException x) {
// OK, some other kind of dep, just ignore
continue;
}
} else { //m2.matches()
groupId = m2.group(1);
artifactId = m2.group(2);
try {
version = new VersionNumber(m2.group(6));
} catch (IllegalArgumentException x) {
// OK, some other kind of dep, just ignore
continue;
}
}
if (groupId.equals("org.jenkins-ci.main") && artifactId.equals("jenkins-core")) {
coreDep = version;
} else if (groupId.equals("org.jenkins-ci.plugins")) {
if(m2.matches()) {
pluginDepsTest.put(artifactId, version);
} else {
pluginDeps.put(artifactId, version);
}
} else if (groupId.equals("org.jenkins-ci.main") && artifactId.equals("maven-plugin")) {
if(m2.matches()) {
pluginDepsTest.put(artifactId, version);
} else {
pluginDeps.put(artifactId, version);
}
} else if (groupId.equals(pluginGroupIds.get(artifactId))) {
if(m2.matches()) {
pluginDepsTest.put(artifactId, version);
} else {
pluginDeps.put(artifactId, version);
}
}
}
}
} finally {
Files.delete(tmp.toPath());
}
System.out.println("Analysis: coreDep=" + coreDep + " pluginDeps=" + pluginDeps + " pluginDepsTest=" + pluginDepsTest);
if (coreDep != null) {
Map<String,VersionNumber> toAdd = new HashMap<>();
Map<String,VersionNumber> toReplace = new HashMap<>();
Map<String,VersionNumber> toAddTest = new HashMap<>();
Map<String,VersionNumber> toReplaceTest = new HashMap<>();
overridenPlugins.forEach(plugin -> {
toReplace.put(plugin.getName(), plugin.getVersion());
toReplaceTest.put(plugin.getName(), plugin.getVersion());
if (plugin.getGroupId() != null) {
if (pluginGroupIds.containsKey(plugin.getName())) {
if (!plugin.getGroupId().equals(pluginGroupIds.get(plugin.getName()))) {
System.err.println("WARNING: mismatch between detected and explicit group ID for " + plugin.getName());
}
} else {
pluginGroupIds.put(plugin.getName(), plugin.getGroupId());
}
}
});
for (String split : splits) {
String[] pieces = split.split(" ");
String plugin = pieces[0];
if (splitCycles.contains(thisPlugin + ' ' + plugin)) {
System.out.println("Skipping implicit dep " + thisPlugin + " → " + plugin);
continue;
}
VersionNumber splitPoint = new VersionNumber(pieces[1]);
VersionNumber declaredMinimum = new VersionNumber(pieces[2]);
if (coreDep.compareTo(splitPoint) < 0 && new VersionNumber(coreVersion).compareTo(splitPoint) >=0 && !pluginDeps.containsKey(plugin)) {
Plugin bundledP = otherPlugins.get(plugin);
if (bundledP != null) {
VersionNumber bundledV;
try {
bundledV = new VersionNumber(bundledP.version);
} catch (NumberFormatException x) { // TODO apparently this does not handle `1.0-beta-1` and the like?!
System.out.println("Skipping unparseable dep on " + bundledP.name + ": " + bundledP.version);
continue;
}
if (bundledV.isNewerThan(declaredMinimum)) {
toAdd.put(plugin, bundledV);
continue;
}
}
toAdd.put(plugin, declaredMinimum);
}
}
List<String> convertFromTestDep = new ArrayList<>();
checkDefinedDeps(pluginDeps, toAdd, toReplace, otherPlugins, new ArrayList<>(pluginDepsTest.keySet()), convertFromTestDep);
pluginDepsTest.putAll(difference(pluginDepsTest, toAdd));
pluginDepsTest.putAll(difference(pluginDepsTest, toReplace));
checkDefinedDeps(pluginDepsTest, toAddTest, toReplaceTest, otherPlugins);
// Could contain transitive dependencies which were part of the plugin's dependencies or to be added
toAddTest = difference(pluginDeps, toAddTest);
toAddTest = difference(toAdd, toAddTest);
if (!toAdd.isEmpty() || !toReplace.isEmpty() || !toAddTest.isEmpty() || !toReplaceTest.isEmpty()) {
System.out.println("Adding/replacing plugin dependencies for compatibility: " + toAdd + " " + toReplace + "\nFor test: " + toAddTest + " " + toReplaceTest);
pom.addDependencies(toAdd, toReplace, toAddTest, toReplaceTest, pluginGroupIds, convertFromTestDep);
}
// TODO(oleg_nenashev): This is a hack, logic above should be refactored somehow (JENKINS-55279)
// Remove the self-dependency if any
pom.removeDependency(pluginGroupIds.get(thisPlugin), thisPlugin);
}
else {
// bad, we should always find a core dependency!
throw new PomTransformationException("No jenkins core dependency found, aborting!", new Throwable());
}
}
worked (db23f04 was a long time ago!) but if there is some PCT failure in junit related to plugins split from core then let us deal with it properly, in hpi:resolve-test-dependencies.

a weird behavior detected on testing junit plugin

https://github.com/jenkinsci/bom/runs/8466852791 is passing so you are in some other situation but what exactly?

@imonteroperez
Copy link
Contributor Author

but if there is some PCT failure in junit related to plugins split from core then let us deal with it properly

IIUC @raul-arabaolaza pointed the issue is not only for junit. On the other hand, he was asking about why pom files are not being transformed, and here is the reason.

@basil
Copy link
Member

basil commented Sep 21, 2022

The POM rewriting functionality was intentionally replaced with logic in maven-hpi-plugin to rewrite the dependency tree in memory. If there is a problem with how the rewritten logic deals with split plugins, a fix should be made there. The new logic logs heavily (even more so when Maven is run in debug mode), so it should be possible to analyze the logs and see if the new logic is behaving as expected or not.

@imonteroperez
Copy link
Contributor Author

The POM rewriting functionality was intentionally replaced with logic in maven-hpi-plugin to rewrite the dependency tree in memory

Great then, thanks for clarifying! closing this PR and letting @raul-arabaolaza to re-open and/or submit an issue in case of need. Thanks!

@jglick jglick mentioned this pull request Dec 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants