Date: Thu, 28 Mar 2024 13:21:43 +0000 (UTC)
Message-ID: <1653144444.6455.1711632103004@ip-10-10-7-29.ec2.internal>
Subject: Exported From Confluence
MIME-Version: 1.0
Content-Type: multipart/related;
boundary="----=_Part_6454_1248754180.1711632103002"
------=_Part_6454_1248754180.1711632103002
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Content-Location: file:///C:/exported.html
Introduction
The current implementation of Grouper uses Ant tasks=
to build its artifacts, relying on third-party dependencies being supplied=
along with each submodule. The versions of these jar files are not always =
clear, as the version is often missing from the filename, and the MANIFEST.=
MF file is sometimes missing an Implementation-Version property or it is in=
accurate. When comparing these files versus official sources (e.g. Maven re=
positories), some of them are identical to published versions, some are clo=
se matches, and some can't be matched to any known version. One goal of thi=
s project is to replace all official jars with ones from known provenance. =
Files should always include the version number in the filename, to make it =
much easier to identify the version, and to make it easier to track down in=
public locations.
In addition, some of these supplied jars are many ve=
rsions behind the current release. Where possible, we should upgrade to the=
latest version that is still compatible without breaking existing function=
ality.
The identification compares local jars against versi=
ons uploaded to the Maven Central Repository (http://sea=
rch.maven.org/). The versions for supplied jars are taken first from th=
e MANIFEST.MF property Implementation-Version if available, then from the v=
ersion in the filename, otherwise any other clues in the MANIFEST file of t=
he jar. If the version can be identified, a guess at a corresponding versio=
n in the Maven repository is compared, The strongest match is an identical =
checksum. The second best is that all of the class files are a binary match=
, disregarding Java source code, Javadoc html files, testing classes, or ot=
her miscellaneous files. If there are extra or missing classes, it suggests=
that either the alleged version is misidentified, or the supplied jar bund=
les multiple libraries into a single jar file. If there still is no good ma=
tch, the supplied file is left alone, as it will likely be upgraded later.<=
/p>
Testing upgraded jars will first involve running the=
full test suite from the source code JUnit tests (2100+ tests over 5 hours=
), to verify no regressions are introduced. Then the Grouper UI, Web Servic=
es, and other submodules will need to be broadly tested to confirm all func=
tions continue to work.
Status as of Grouper 2.4.0 Release (in 2018)
As of Grouper release 2.4.0, the libraries have been updated in the offi=
cial distribution. The equivalent dependency versions in Maven pom.xml file=
s have also been updated to match as close as possible. Grouper 2.3.0 or ea=
rlier versions have not been updated, even with patches, except to remedy s=
pecific issues.
Grouper API
Phase 1 - =
Identify jars identical or close matches with official versions, rename or =
replace with official one if needed
Of the approximately 70 jars in the lib/grouper, lib=
/jdbcSamples, and lib/jetty directories, a good number already match both t=
he checksum and the filename, so that nothing needs to be done.
- antlr-2.7.7.jar
- c3p0-0.9.5.2.jar
- commons-lang3-3.5.jar
- cron-parser-core-2.9.jar
- dom4j-1.6.1.jar
- ehcache-core-2.4.3.jar
- groovy-all-2.5.0-beta-2.jar
- hibernate-c3p0-5.0.4.Final.jar
- hibernate-commons-annotations-5.0.0.Final.jar
- hibernate-core-5.0.4.Final.jar
- hibernate-ehcache-5.0.4.Final.jar
- hibernate-jpa-2.1-api-1.0.0.Final.jar
- jandex-2.0.0.CR1.jar
- javassist-3.18.1-GA.jar
- jboss-logging-3.3.0.Final.jar
- jline-2.14.5.jar
- joda-time-2.9.7.jar
- mchange-commons-java-0.2.11.jar
- slf4j-api-1.6.1.jar
- slf4j-log4j12.jar (near match)
Some others match the checksum of a published file, =
but are missing the version number from the filename. These can be renamed:=
- bsh.jar -> bsh-2.0b4.jar
- vt-ldap.jar -> vt-ldap-3.3.9.jar
- commons-math.jar -> commons-math-1.1.jar
- jakarta-oro.jar -> oro-2.0.8.jar
- commons-codec.jar -> commons-codec-1.3.jar
- commons-betwixt.jar -> commons-betwixt-0.8.jar=
li>
- commons-cli.jar -> commons-cli-1.4.jar
- commons-httpclient.jar -> commons-httpclient-3.0=
.jar
- commons-discovery.jar -> commons-discovery-0.4.j=
ar
- jetty/servlet-api.jar -> servlet-api-2.5-2008121=
1.jar
Others don't match the checksum, but have identical =
.class objects within the files, so wouldn't be affected by updating to the=
official version. The file differences are solely in the existence of java=
doc or source code, or in other properties in the manifest file.
- ant.jar -> ant-1.7.1.jar
- asm-util.jar -> asm-util-3.3.1.jar
- asm.jar -> asm-3.3.1.jar
- backport-util-concurrent.jar -> backport-util-co=
ncurrent-3.0.jar
- commons-beanutils.jar -> commons-beanutils-1.7.0=
.jar
- commons-collections.jar -> commons-collections-3=
.2.1.jar
- commons-io.jar -> commons-io-1.4.jar
- commons-lang.jar -> commons-lang-2.6.jar
- commons-logging.jar -> commons-logging-1.1.1.jar=
- ddlUtils.jar -> ddlutils-1.0.jar
- ezmorph.jar -> ezmorph-1.0.6.jar
- json-lib.jar -> json-lib-2.3-jdk15.jar
- jsr107cache.jar -> jsr107cache-1.0.jar
- jug.jar -> jug-1.1.2.jar
- log4j.jar -> log4j-1.2.17.jar
- quartz.jar -> quartz-2.2.2.jar
- smack.jar -> smack-3.1.0.jar
- xpp3_min.jar -> xpp3_min-1.1.4c.jar
- xstream.jar -> xstream-1.3.jar
- jetty/jetty.jar -> jetty-6.1.24.jar
- jetty/jetty-util.jar -> jetty-util-6.1.24.jar
- jdbcSamples/mysql-connector-java-bin.jar -> mysq=
l-connector-java-5.1.38.jar
- jdbcSamples/postgresql.jar -> postgresql-9.3-110=
1-jdbc3.jar
The remaining files either didn't have its version a=
vailable in the Maven repository, or had significant differences with the p=
ublished version, so more analysis is needed.
- c3p0-oracle-thin-extras-0.9.5.2.jar ("0.9.5.2" not =
in Maven; vs. "0.9.5" just two classes differ)
- commons-jexl.jar (15 classes differ)
- jamon.jar ("2.7" not available; vs. 2.75, 98 classe=
s differ)
- activation.jar (a few classes differ)
- odmg.jar (all classes differ, though file sizes are=
the same)
- mailapi.jar (version "1.3.2" not available; vs. 1.4=
.2, many classes differ, plus newer missing a few classes)
- smtp.jar (version "1.3.2" not available; version 1.=
4.4 may be compatible, newer adds support for ssl, only a few classes chang=
ed)
- p6spy.jar (newer missing many classes)
- commons-digester.jar (supplied jar has both Digeste=
r and Digester3 packages; would need to supply both jars if replacing)
- jta.jar ("1.0.1B" not available ; vs. 1.1, 17 class=
es differ)
- jdbcSamples/hsqldb.jar (classes differ vs. 2.3.3, s=
upplied manifest version "private-2015/07/02-14:15:34" by company "ft")
- jdbcSamples/ojdbc6_g.jar (no official version avail=
able, due to licensing)
- jdbcSamples/sqljdbc4.jar (no official version avail=
able, due to licensing)
Phase 2 -=
Identify unneeded jars
It's feasible that, over time, a library may have be=
en added that is no longer needed due to code changes. If a library can be =
safely removed from the distribution, it means a smaller distribution file =
size, and is one less library to be maintained and updated. If a library is=
not needed by the Grouper API library but is needed in another subproject,=
it should be moved out of the API library folder and into the other projec=
t's folder.
The Grouper source code has certain direct dependenc=
ies on third-party library jars. Many of these libraries have dependencies =
of their own. Two methods that can be used to discover these dependencies a=
re the Java jdeps command and Maven packaging operations. While these metho=
ds can reliably identify compile-phase libraries, it is harder to know abou=
t classes that are needed at runtime.
- asm-3.3.1.jar
- asm-util-3.3.1.jar
- commons-betwixt-0.8.jar
- (NO THIS IS NEEDED BY MANY CLASSES) commons-lang3-3=
.5.jar
- invoker.jar (this was only used pre-Java 1.4 to loa=
d a wildcard directory of jars for GSH; it only is referenced in comments i=
n the GSH boot script)
- (NO THIS IS NEEDED TO RUN TESTS) jamon.jar
- joda-time-2.9.7.jar
- jsr107cache-1.0.jar
- odmg.jar
- (NO THIS IS NEEDED AT RUNTIME BY ddlUtils) oro-2.0.=
8.jar
- smtp.jar (javax mail =E2=80=93 not mailapi =E2=
=80=93 plus activation.jar can replace the necessary smtp functions)
- xpp3_min-1.1.4c.jar (its classes are all duplicated=
in smack.jar)
Phase 3 - Where possible, upgrade remaining jars to =
the latest version
Already the latest version:
- vt-ldap-3.3.9.jar
- commons-cli-1.4.jar
- hibernate-jpa-2.1-api-1.0.0.Final.jar
- jline-2.14.5.jar
- (removed, actually) joda-time-2.9.7.jar
- groovy-all-2.5.0-beta-2.jar
- c3p0-0.9.5.2.jar
correctly named but not current version -> git rm=
and git add the new one:
- bsh (2.0b4.jar -> 2.0b5.jar)
- commons-codec (1.3 -> 1.11)
- commons-discovery (0.4 -> 0.5) (removed later as=
potentially unneeded)
- commons-httpclient (3.0 -> 3.1)
- commons-lang3 (3.5 -> 3.7)
- commons-math (1.1 -> 1.2)
- cron-parser-core (2.9 -> 3.4)
- dom4j (1.6.1 -> 2.1.0)
- ehcache-core (2.4.3 -> 2.6.11)
- hibernate-c3p0 (5.0.4.Final.jar -> 5.0.12.Final.=
jar)
- hibernate-commons-annotations (5.0.0.Final.jar ->=
; 5.0.1.Final.jar)
- hibernate-core (5.0.4.Final.jar -> 5.0.12.Final.=
jar)
- hibernate-ehcache (5.0.4.Final.jar -> 5.0.12.Fin=
al.jar)
- jandex (2.0.0.CR1.jar -> 2.0.4.Final.jar)
- javassist (3.18.1-GA.jar -> 3.22.0-GA.jar)
- jboss-logging (3.3.0.Final.jar -> 3.3.1.Final.ja=
r)
- mchange-commons-java (0.2.11 -> 0.2.14)
- slf4j-api (1.6.1 -> 1.7.25)
- slf4j-log4j12 (1.6.2 -> 1.7.25)
Files at the latest version, allegedly the correct n=
ame but not an official maven version -> replace with official one and u=
pdate in git:
- commons-lang-2.6.jar
- ddlutils-1.0.jar
- ezmorph-1.0.6.jar
- jug-1.1.2.jar (did not attempt to upgrade to the 2.=
* series) ? - log4j-1.2.17.jar
Files that weren't a good match to Maven anyway, git=
remove and replace
- activation (1.0.2? -> 1.1.1.jar)
- ant (1.7.1 -> 1.10.1.jar)
- backport-util-concurrent (3.0 -> 3.1.jar)
- c3p0-oracle-thin-extras (0.9.5.2 -> 0.9.5.jar)=
li>
- commons-beanutils (~1.7.0? -> 1.9.3.jar)
- commons-collections (3.2.1 -> 3.2.2.jar)
- commons-digester (3.2? -> 3.2.jar)
- commons-jexl (2.0.1? -> 2.1.1.jar)
- commons-logging (1.1.1 -> 1.2.jar)
- jamon (2.7? -> 2.81.jar)
- json-lib (2.3-jdk15 -> 2.4-jdk15.jar)
- jta (1.0.1B? -> 1.1.jar)
- mailapi (1.3.2? -> 1.4.3.jar)
- p6spy (1.1? -> 3.6.0.jar)
- quartz (2.2.2 -> 2.3.0.jar)
- smack (3.1.0? -> 3.1.0.jar)
- xstream (1.3 -> 1.4.10.jar)
Note that the following could not be upgraded to the=
latest version due to compile or test issues:
- antlr (2.7.7 -> 3.5.2) is not compatible without=
Grouper code changes
- commons-math (1.1 -> 1.2) fails to compile
- hibernate >5.0.12 had various issues during comp=
ile or testing; will tentatively keep at 5.0.12 wich is the latest of the 5=
.0 series
- ehcache-core (2.4.3 -> 2.6.11) had issues; will =
tentatively keep at 2.4.8 which is the latest of the 2.4.* series
- commons-io (1.4 -> 2.6) had compatibility issues=
with the InstrumentationThread class
Grouper UI
Phase 1 - Identify jars identical or close matches with offic=
ial versions, rename or replace with official one if needed
Identical checksums, can be renamed:
- ant-contrib.jar -> ant-contrib-1.0b3.jar
- struts-el.jar -> struts-el-1.2.4.jar
- xml-apis.jar -> xml-apis-1.3.02.jar
Don't match checksum, but *.class files identical (difference is *.java,=
javadoc, or manifest)
- commons-fileupload.jar -> commons-fileupload-1.2.1.jar
- jstl.jar -> jstl-1.1.2.jar
- opencsv.jar -> opencsv-1.8.jar
- xercesImpl.jar -> xercesImpl-2.7.1.jar
Most classes match, but a few minor differences:
- struts.jar -> struts-1.2.4.jar (Maven version has a few extra classe=
s)
- standard.jar -> standard-1.1.2.jar (supplied jar has extra tld and s=
ome test docs)
Phase 2 - Identify unneeded jars
Using Java jdeps command, along with explicit dependencies listed for Ma=
ven versions of included jars, identify libraries that can potentially be r=
emoved:
- jakarta-oro.jar -> removed
- serializer.jar -> removed
- taglibs-datetime.jar -> removed*
- taglibs-request.jar -> removed*
- xalan.jar -> removed
It was later discovered that taglibs-datetime and taglibs-request are ne=
eded by Struts 1 and the admin UI. They are being left out since the admin =
UI will eventually be removed.
Phase 3 - Where possible, up=
grade remaining jars to the latest version
- commons-fileupload 1.2.1 -> 1.3.3
- csrfguard.jar 3.1.0-SNAPSHOT? -> 3.1.0
- jstl 1.1.2 -> 1.2.1
- opencsv 1.8 -> 2.4
- xercesImpl 2.7.1 -> 2.11.0
- xml-apis 1.3.02 -> 1.4.01
Also, the supplied servlet.jar seems to be the fully implemented servlet=
library normally supplied by the servlet container. Thus, it was replaced =
by the API stubs servlet-api and jsp-api
- servlet.jar (replaced with servlet-api and jsp-api)
- servlet-api 2.5 (from the API jetty/ folder) -> 3.1.0 (moved to the =
API grouper folder)
- jsp-api 2.0 -> 2.3.2-b02
Group=
er WS
As most of the libraries in this project already match versions from the=
Maven repository, little needed to be done here. Upgrading the exising jar=
s, along with analysis of potentially unneeded libraries, weren't done at t=
his time.
- commons-lang.jar -> commons-lang-2.1.jar
- junit.jar -> junit-3.8.1.jar OR junit-4.1.jar depending on submodule=
- ant-contrib.jar -> ant-contrib-1.0b3.jar
- servlet.jar -> servlet-api-2.5.jar
- commons-net.jar -> commons-net-1.3.0.jar
- stax-1.2.0.jar -> refreshed from Maven
Subject=
The subject module has been combined into the grouper module.
G=
rouperClient
The grouper client is aimed at creating an executable jar. Thus, there a=
re no external libraries involved.
------=_Part_6454_1248754180.1711632103002--