Hey Leute, ich versuche mit sbt-assembly eine JAR mit Abhängigkeiten zu bauen. Aber ich stoße immer wieder auf diesen Fehler. Ich habe mehrere verschiedene Dinge ausprobiert, aber ich lande hier. Ich bin ziemlich neu bei SBT und wollte hier etwas Hilfe bekommen. Hier sind die Dateien build.sbt & assembly.sbt.

build.sbt

seq(assemblySettings: _*)

name := "StreamTest"

version := "1.0"

scalaVersion := "2.10.4"

libraryDependencies += "org.apache.spark" %% "spark-core" % "1.1.0"

libraryDependencies += "org.apache.spark" % "spark-streaming-kinesis-asl_2.10" % "1.1.0"

project / assembly.sbt

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.11.2")

Wenn ich den Befehl sbt assembly ausführe, wird dieser Fehler unten angezeigt.

[info] Including: joda-time-2.5.jar
[info] Checking every *.class/*.jar file's SHA-1.
[info] Merging files...
[warn] Merging 'about_files/LICENSE.txt' with strategy 'rename'
[warn] Merging 'about_files/NOTICE.txt' with strategy 'rename'
[warn] Merging 'META-INF/NOTICE.txt' with strategy 'rename'
[warn] Merging 'META-INF/NOTICE' with strategy 'rename'
[warn] Merging 'org/xerial/snappy/native/README' with strategy 'rename'
[warn] Merging 'META-INF/maven/org.xerial.snappy/snappy-java/LICENSE' with strategy 'rename'
[warn] Merging 'META-INF/license' with strategy 'rename'
[warn] Merging 'about.html' with strategy 'rename'
[warn] Merging 'META-INF/LICENSE.txt' with strategy 'rename'
[warn] Merging 'META-INF/README.txt' with strategy 'rename'
[warn] Merging 'LICENSE.txt' with strategy 'rename'
[warn] Merging 'META-INF/LICENSE' with strategy 'rename'
[warn] Merging 'META-INF/DEPENDENCIES' with strategy 'discard'
java.lang.RuntimeException: deduplicate: different file contents found in the following:
/Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.transaction/orbits/javax.transaction-1.1.1.v201105210645.jar:META-INF/ECLIPSEF.RSA
/Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.servlet/orbits/javax.servlet-3.0.0.v201112011016.jar:META-INF/ECLIPSEF.RSA
/Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.mail.glassfish/orbits/javax.mail.glassfish-1.4.1.v201005082020.jar:META-INF/ECLIPSEF.RSA
/Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.activation/orbits/javax.activation-1.1.0.v201105071233.jar:META-INF/ECLIPSEF.RSA
    at sbtassembly.Plugin$Assembly$.sbtassembly$Plugin$Assembly$applyStrategy$1(Plugin.scala:253)
    at sbtassembly.Plugin$Assembly$anonfun$15.apply(Plugin.scala:270)
    at sbtassembly.Plugin$Assembly$anonfun$15.apply(Plugin.scala:267)
    at scala.collection.TraversableLike$anonfun$flatMap$1.apply(TraversableLike.scala:251)
    at scala.collection.TraversableLike$anonfun$flatMap$1.apply(TraversableLike.scala:251)
    at scala.collection.Iterator$class.foreach(Iterator.scala:727)
    at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
    at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
    at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
    at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251)
    at scala.collection.AbstractTraversable.flatMap(Traversable.scala:105)
    at sbtassembly.Plugin$Assembly$.applyStrategies(Plugin.scala:272)
    at sbtassembly.Plugin$Assembly$.x$4$lzycompute$1(Plugin.scala:172)
    at sbtassembly.Plugin$Assembly$.x$4$1(Plugin.scala:170)
    at sbtassembly.Plugin$Assembly$.stratMapping$lzycompute$1(Plugin.scala:170)
    at sbtassembly.Plugin$Assembly$.stratMapping$1(Plugin.scala:170)
    at sbtassembly.Plugin$Assembly$.inputs$lzycompute$1(Plugin.scala:214)
    at sbtassembly.Plugin$Assembly$.inputs$1(Plugin.scala:204)
    at sbtassembly.Plugin$Assembly$.apply(Plugin.scala:230)
    at sbtassembly.Plugin$Assembly$anonfun$assemblyTask$1.apply(Plugin.scala:373)
    at sbtassembly.Plugin$Assembly$anonfun$assemblyTask$1.apply(Plugin.scala:370)
    at scala.Function1$anonfun$compose$1.apply(Function1.scala:47)
    at sbt.$tilde$greater$anonfun$u2219$1.apply(TypeFunctions.scala:40)
    at sbt.std.Transform$anon$4.work(System.scala:63)
    at sbt.Execute$anonfun$submit$1$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.Execute$anonfun$submit$1$anonfun$apply$1.apply(Execute.scala:226)
    at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
    at sbt.Execute.work(Execute.scala:235)
    at sbt.Execute$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.Execute$anonfun$submit$1.apply(Execute.scala:226)
    at sbt.ConcurrentRestrictions$anon$4$anonfun$1.apply(ConcurrentRestrictions.scala:159)
    at sbt.CompletionService$anon$2.call(CompletionService.scala:28)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
[error] (*:assembly) deduplicate: different file contents found in the following:
[error] /Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.transaction/orbits/javax.transaction-1.1.1.v201105210645.jar:META-INF/ECLIPSEF.RSA
[error] /Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.servlet/orbits/javax.servlet-3.0.0.v201112011016.jar:META-INF/ECLIPSEF.RSA
[error] /Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.mail.glassfish/orbits/javax.mail.glassfish-1.4.1.v201005082020.jar:META-INF/ECLIPSEF.RSA
[error] /Users/user/.ivy2/cache/org.eclipse.jetty.orbit/javax.activation/orbits/javax.activation-1.1.0.v201105071233.jar:META-INF/ECLIPSEF.RSA
[error] Total time: 23 s, completed Nov 28, 2014 9:32:53 PM

sbt-version

0.13.6

BEARBEITE

Nun nachdem ich mich umgesehen habe, habe ich eine weitere Änderung vorgenommen, um Abhängigkeiten auszuschließen, als Teil einer weiteren Frage zum Stackoverflow.

Updated build.sbt

seq(assemblySettings: _*)

name := "StreamTest"

version := "1.0"

scalaVersion := "2.10.4"

libraryDependencies += "org.apache.spark" %% "spark-core" % "1.1.0"

libraryDependencies += "org.apache.spark" % "spark-streaming-kinesis-asl_2.10" % "1.1.0"

libraryDependencies ++= Seq(
    exclude("org.eclipse.jetty.orbit", "javax.servlet").
    exclude("org.eclipse.jetty.orbit", "javax.transaction").
    exclude("org.eclipse.jetty.orbit", "javax.mail").
    exclude("org.eclipse.jetty.orbit", "javax.activation").
    exclude("commons-beanutils", "commons-beanutils-core").
    exclude("commons-collections", "commons-collections").
    exclude("commons-collections", "commons-collections").
    exclude("com.esotericsoftware.minlog", "minlog")
)

Wenn ich den Assembly-Befehl erneut ausführe, ist dies der Fehler, den ich erhalte.

build.sbt:14: error: not found: value exclude
    exclude("org.eclipse.jetty.orbit", "javax.servlet").
    ^

EDIT 2:

Updated build.sbt

import AssemblyKeys._

seq(assemblySettings: _*)

name := "SparkStreamingKinesis"

version := "1.0"

scalaVersion := "2.10.4"

libraryDependencies += "org.apache.spark" %% "spark-core" % "1.1.0"

libraryDependencies += "org.apache.spark" % "spark-streaming-kinesis-asl_2.10" % "1.1.0"

assemblyMergeStrategy in assembly := {
    case PathList(ps @ _*) if ps.last endsWith ".RSA" => MergeStrategy.first
    case x =>
       val oldStrategy = (assemblyMergeStrategy in assembly).value
       oldStrategy(x)
}

Nach dem Update ist hier der Fehler, den ich erhalte. Ich habe versucht, assemblyMergeStrategy zu importieren, aber es sieht nicht nach einer Klasse aus, die ich importieren kann.

build.sbt:21: error: not found: value assemblyMergeStrategy
assemblyMergeStrategy in assembly := {
^
[error] Type error in expression