Hammering Nexus 3.14

first, create an entry in your settings.xml for mvn to be able to authenticate in nexus

<server>
<id>nexus</id>
<username>admin</username>
<password>admin123</password>
</server>

then write this testnexus.sh script:

NEXUSHOME=/home/centos/nexus314
NEXUSLOG=$NEXUSHOME/sonatype-work/nexus3/log/nexus.log
rm $NEXUSLOG

cd $NEXUSHOME/nexus-3.14.0-04/bin

for i in `seq 1 1000`;
do
 echo "Iteration number " $i
 #make sure nexus is not running before you start it
 ./nexus stop
 ./nexus start
 while [ ! -f $NEXUSLOG ] ;
 do
  sleep 2
 done

 echo "logfile found"
 tail -f $NEXUSLOG | while read LOGLINE
 do
  [[ "$LOGLINE" == *"Started Sonatype Nexus OSS 3.14.0-04"* ]] && pkill -P $$ tail
 done
 #now that it's up, let's wait for some time

 echo "nexus started"
 NEXUSPID=`ps -e -o pid,command | grep org.sonatype.nexus.karaf.NexusMain | grep -v grep | awk '{print $1}'`
 echo NEXUSPID = $NEXUSPID
 #wait for a random time before we kill
 sleep $[ ( $RANDOM % 100 )  + 10 ]
 ps -ef | grep nexus
 echo killing $NEXUSPID
 kill -9 $NEXUSPID
 mv $NEXUSLOG  ${NEXUSLOG}.${i}
done

and here the 2 scripts to upload continuously stuff to nexus

testnexusupload1.sh

NEXUSHOME=/home/centos/nexus314

curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.pom
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.jar
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.md5
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.sha1
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.pom.sha1
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.pom.md5
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.jar.sha1
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.jar.md5


mvn deploy:deploy-file -DgroupId=commons-beanutils -DartifactId=commons-beanutils -Dversion=1.9.2 -DgeneratePom=true -Dpackaging=jar -DrepositoryId=nexus -Durl=http://localhost:8081/repository/maven-releases -Dfile=$NEXUSHOME/nexus-3.14.0-04/system/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.jar

testnexusupload2.sh

NEXUSHOME=/home/centos/nexus314

curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-codec/commons-codec/1.10/commons-codec-1.10.pom
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-codec/commons-codec/1.10/commons-codec-1.10.jar
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-codec/commons-codec/1.10/commons-codec-1.10.md5
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-codec/commons-codec/1.10/commons-codec-1.10.sha1
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-codec/commons-codec/1.10/commons-codec-1.10.pom.sha1
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-codec/commons-codec/1.10/commons-codec-1.10.pom.md5
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-codec/commons-codec/1.10/commons-codec-1.10.jar.sha1
curl -X DELETE -u admin:admin123 http://localhost:8081/repository/maven-releases/commons-codec/commons-codec/1.10/commons-codec-1.10.jar.md5


mvn deploy:deploy-file -DgroupId=commons-codec -DartifactId=commons-codec -Dversion=1.10 -DgeneratePom=true -Dpackaging=jar -DrepositoryId=nexus -Durl=http://localhost:8081/repository/maven-releases -Dfile=$NEXUSHOME/nexus-3.14.0-04/system/commons-codec/commons-codec/1.10/commons-codec-1.10.jar

testnexusupload3.sh (this will create always new jars)

NEXUSHOME=/home/centos/nexus314
for i in `seq 1 1000000`;
do
 mvn deploy:deploy-file -DgroupId=commons-beanutils -DartifactId=commons-beanutils -Dversion=1.10.${i} -DgeneratePom=true -Dpackaging=jar -DrepositoryId=nexus -Durl=http://localhost:8081/repository/maven-releases -Dfile=$NEXUSHOME/nexus-3.14.0-04/system/commons-beanutils/commons-beanutils/1.9.2/commons-beanutils-1.9.2.jar

 sleep 2
done

nohup ./testnexusupload1.sh > ./testnexusupload1.log 2>&1 &
nohup ./testnexusupload2.sh > ./testnexusupload2.log 2>&1 &
nohup ./testnexusupload3.sh > ./testnexusupload3.log 2>&1 &

or

setsid ./testnexusupload1.sh > ./testnexusupload1.log 2>&1 &
setsid ./testnexusupload2.sh > ./testnexusupload2.log 2>&1 &
setsid ./testnexusupload3.sh > ./testnexusupload3.log 2>&1 &


This way with a continuous cycle of restarts I can test the stability of the repository.
If the DB or shards are corrupted we will find out...



In fact, after some iteration I keep getting this error at startup:




2018-10-28 19:04:49,571+0100 INFO [elasticsearch[C1896386-DD1B2DA2-9E77236C-5051BC20-E1D39AE4][clusterService#updateTask][T#1]] *SYSTEM org.elasticsearch.cluster.metadata - [C1896386-DD1B2DA2-9E77236C-5051BC20-E1D39AE4] [2e9a1e67e8a325bcd6ee9f6790ff6c769e791d56] creating index, cause [api], templates [], shards [1]/[0], mappings [component]
2018-10-28 19:04:49,955+0100 INFO [elasticsearch[C1896386-DD1B2DA2-9E77236C-5051BC20-E1D39AE4][clusterService#updateTask][T#1]] *SYSTEM org.elasticsearch.cluster.routing.allocation - [C1896386-DD1B2DA2-9E77236C-5051BC20-E1D39AE4] Cluster health status changed from [RED] to [GREEN] (reason: [shards started [[2e9a1e67e8a325bcd6ee9f6790ff6c769e791d56][0]] ...]).
2018-10-28 19:04:50,065+0100 INFO [quartz-2-thread-1] *SYSTEM org.sonatype.nexus.repository.search.SearchFacetImpl - Rebuilding index of repository maven-central
2018-10-28 19:04:50,501+0100 INFO [elasticsearch[C1896386-DD1B2DA2-9E77236C-5051BC20-E1D39AE4][clusterService#updateTask][T#1]] *SYSTEM org.elasticsearch.cluster.metadata - [C1896386-DD1B2DA2-9E77236C-5051BC20-E1D39AE4] [73ae44bc066b6a7a33b4435641d8229b9b66495a] creating index, cause [api], templates [], shards [1]/[0], mappings [component]
2018-10-28 19:04:50,943+0100 ERROR [Thread-44 <command>sql.select from component where bucket = :bucket</command>] *SYSTEM com.orientechnologies.orient.core.storage.impl.local.paginated.OLocalPaginatedStorage - Exception `32DA9B7B` in storage `plocal:/home/centos/nexus314/sonatype-work/nexus3/db/component`: 2.2.36 (build d3beb772c02098ceaea89779a7afd4b7305d3788, branch 2.2.x)
com.orientechnologies.orient.core.exception.OCommandExecutionException: Error on execution of command: sql.select from component where bucket = :bucket
DB name="component"
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.executeCommand(OAbstractPaginatedStorage.java:3421)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.command(OAbstractPaginatedStorage.java:3318)
at com.orientechnologies.orient.core.sql.query.OSQLQuery.run(OSQLQuery.java:78)
at com.orientechnologies.orient.core.sql.query.OSQLAsynchQuery.run(OSQLAsynchQuery.java:74)
at com.orientechnologies.orient.core.query.OQueryAbstract.execute(OQueryAbstract.java:33)
at com.orientechnologies.orient.core.sql.query.OSQLNonBlockingQuery$1.run(OSQLNonBlockingQuery.java:283)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException: null
at com.orientechnologies.orient.core.serialization.serializer.binary.impl.index.OCompositeKeySerializer.deserializeFromByteBufferObject(OCompositeKeySerializer.java:347)
at com.orientechnologies.orient.core.serialization.serializer.binary.impl.index.OCompositeKeySerializer.deserializeFromByteBufferObject(OCompositeKeySerializer.java:46)
at com.orientechnologies.orient.core.storage.impl.local.paginated.base.ODurablePage.deserializeFromDirectMemory(ODurablePage.java:166)
at com.orientechnologies.orient.core.index.sbtree.local.OSBTreeBucket.getKey(OSBTreeBucket.java:266)
at com.orientechnologies.orient.core.index.sbtree.local.OSBTreeBucket.find(OSBTreeBucket.java:134)
at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.findBucket(OSBTree.java:1679)
at com.orientechnologies.orient.core.index.sbtree.local.OSBTree.access$2800(OSBTree.java:69)
at com.orientechnologies.orient.core.index.sbtree.local.OSBTree$OSBTreeCursorForward.next(OSBTree.java:1991)
at com.orientechnologies.orient.core.index.engine.OSBTreeIndexEngine$OSBTreeIndexCursor.nextEntry(OSBTreeIndexEngine.java:271)
at com.orientechnologies.orient.core.index.OIndexAbstractCursor.hasNext(OIndexAbstractCursor.java:83)
at com.orientechnologies.orient.core.index.OIndexChangesWrapper.hasNext(OIndexChangesWrapper.java:138)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.serialIterator(OCommandExecutorSQLSelect.java:1636)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.fetchFromTarget(OCommandExecutorSQLSelect.java:1585)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.fetchValuesFromIndexCursor(OCommandExecutorSQLSelect.java:2466)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchForIndexes(OCommandExecutorSQLSelect.java:2280)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.searchInClasses(OCommandExecutorSQLSelect.java:1017)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLResultsetAbstract.assignTarget(OCommandExecutorSQLResultsetAbstract.java:203)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.assignTarget(OCommandExecutorSQLSelect.java:527)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.executeSearch(OCommandExecutorSQLSelect.java:509)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLSelect.execute(OCommandExecutorSQLSelect.java:485)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.execute(OCommandExecutorSQLDelegate.java:70)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.executeCommand(OAbstractPaginatedStorage.java:3400)
... 6 common frames omitted