Code

update#1

Code

Merged

YanTangZhai merged 170 commits intoYanTangZhai:masterYanTangZhai/spark:masterfrom apache:masterapache/spark:masterCopy head branch name to clipboardAug 6, 2014

+28,743-4,326

Commits

Commits on Jul 28, 2014

• [SPARK-2523] [SQL] Hadoop table scan bug fixing

  Show description for 2b8d89e
  

  chenghao-intel

  authored and

  marmbrus

  committed
  2b8d89e

  Copy full SHA for 2b8d89e

• [SPARK-2479][MLlib] Comparing floating-point numbers using relative error in UnitTests

  Show description for 255b56f
  

  DB Tsai

  authored and

  mengxr

  committed
  255b56f

  Copy full SHA for 255b56f

• [SPARK-2410][SQL] Merging Hive Thrift/JDBC server (with Maven profile fix)

  Show description for a7a9d14
  

  liancheng

  authored and

  marmbrus

  committed
  a7a9d14

  Copy full SHA for a7a9d14

• Use commons-lang3 in SignalLogger rather than commons-lang

  Show description for 39ab87b
  

  aarondav

  authored and

  rxin

  committed
  39ab87b

  Copy full SHA for 39ab87b

Commits on Jul 29, 2014

• Excess judgment

  Show description for 16ef4d1
  

  watermen

  authored and

  rxin

  committed
  16ef4d1

  Copy full SHA for 16ef4d1

• [SPARK-2580] [PySpark] keep silent in worker if JVM close the socket

  Show description for ccd5ab5
  

  davies

  authored and

  JoshRosen

  committed
  ccd5ab5

  Copy full SHA for ccd5ab5

• [SPARK-791] [PySpark] fix pickle itemgetter with cloudpickle

  Show description for 92ef026
  

  davies

  authored and

  JoshRosen

  committed
  92ef026

  Copy full SHA for 92ef026

• [SPARK-2726] and [SPARK-2727] Remove SortOrder and do in-place sort.

  Show description for 96ba04b
  rxin

  committed
  96ba04b

  Copy full SHA for 96ba04b

• [SPARK-2174][MLLIB] treeReduce and treeAggregate

  Show description for 20424da
  

  mengxr

  authored and

  rxin

  committed
  20424da

  Copy full SHA for 20424da

• Minor indentation and comment typo fixes.

  Show description for fc4d057
  

  staple

  authored and

  rxin

  committed
  fc4d057

  Copy full SHA for fc4d057

• [STREAMING] SPARK-1729. Make Flume pull data from source, rather than the current pu...

  Show description for 800ecff
  

  harishreedharan

  authored and

  tdas

  committed
  800ecff

  Copy full SHA for 800ecff

• [SQL]change some test lists

  Show description for 0c5c6a6
  

  adrian-wang

  authored and

  marmbrus

  committed
  0c5c6a6

  Copy full SHA for 0c5c6a6

• [SPARK-2730][SQL] When retrieving a value from a Map, GetItem evaluates key twice

  Show description for e364348
  

  yhuai

  authored and

  marmbrus

  committed
  e364348

  Copy full SHA for e364348

• [SPARK-2674] [SQL] [PySpark] support datetime type for SchemaRDD

  Show description for f0d880e
  

  davies

  authored and

  marmbrus

  committed
  f0d880e

  Copy full SHA for f0d880e

• [SPARK-2082] stratified sampling in PairRDDFunctions that guarantees exact sample size

  Show description for dc96536
  

  dorx

  authored and

  mengxr

  committed
  dc96536

  Copy full SHA for dc96536

• [SPARK-2393][SQL] Cost estimation optimization framework for Catalyst logical plans & sample usage.

  Show description for c7db274
  

  concretevitamin

  authored and

  marmbrus

  committed
  c7db274

  Copy full SHA for c7db274

Commits on Jul 30, 2014

• MAINTENANCE: Automated closing of pull requests.

  Show description for 2c35666
  pwendell

  committed
  2c35666

  Copy full SHA for 2c35666

• [SPARK-2716][SQL] Don't check resolved for having filters.

  Show description for 39b8193
  marmbrus

  committed
  39b8193

  Copy full SHA for 39b8193

• [SPARK-2631][SQL] Use SQLConf to configure in-memory columnar caching

  Show description for 86534d0
  marmbrus

  committed
  86534d0

  Copy full SHA for 86534d0

• [SPARK-2305] [PySpark] Update Py4J to version 0.8.2.1

  Show description for 22649b6
  

  JoshRosen

  authored and

  mateiz

  committed
  22649b6

  Copy full SHA for 22649b6

• [SPARK-2054][SQL] Code Generation for Expression Evaluation

  Show description for 8446746
  marmbrus

  committed
  8446746

  Copy full SHA for 8446746

• [SPARK-2568] RangePartitioner should run only one job if data is balanced

  Show description for 2e6efca
  

  mengxr

  authored and

  rxin

  committed
  2e6efca

  Copy full SHA for 2e6efca

• [SQL] Handle null values in debug()

  Show description for 077f633
  marmbrus

  committed
  077f633

  Copy full SHA for 077f633

• [SPARK-2260] Fix standalone-cluster mode, which was broken

  Show description for 4ce92cc
  

  andrewor14

  authored and

  pwendell

  committed
  4ce92cc

  Copy full SHA for 4ce92cc

• [SPARK-2179][SQL] Public API for DataTypes and Schema

  Show description for 7003c16
  

  yhuai

  authored and

  marmbrus

  committed
  7003c16

  Copy full SHA for 7003c16

• SPARK-2543: Allow user to set maximum Kryo buffer size

  Show description for 7c5fc28
  

  koertkuipers

  authored and

  pwendell

  committed
  7c5fc28

  Copy full SHA for 7c5fc28

• SPARK-2748 [MLLIB] [GRAPHX] Loss of precision for small arguments to Math.exp, Math.log

  Show description for ee07541
  

  srowen

  authored and

  mengxr

  committed
  ee07541

  Copy full SHA for ee07541

• [SPARK-2521] Broadcast RDD object (instead of sending it along with every task)

  Show description for 774142f
  rxin

  committed
  774142f

  Copy full SHA for 774142f

• [SPARK-2747] git diff --dirstat can miss sql changes and not run Hive tests

  Show description for 3bc3f18
  rxin

  committed
  3bc3f18

  Copy full SHA for 3bc3f18

• Avoid numerical instability

  Show description for e3d85b7
  

  naftaliharris

  authored and

  mengxr

  committed
  e3d85b7

  Copy full SHA for e3d85b7

• [SPARK-2544][MLLIB] Improve ALS algorithm resource usage

  Show description for fc47bb6
  

  witgo

  authored and

  mengxr

  committed
  fc47bb6

  Copy full SHA for fc47bb6

• [SPARK-2746] Set SBT_MAVEN_PROFILES only when it is not set explicitly by the user.

  Show description for ff511ba
  rxin

  committed
  ff511ba

  Copy full SHA for ff511ba

• Wrap FWDIR in quotes.

  rxin

  committed
  f2eb84f

  Copy full SHA for f2eb84f

• Wrap FWDIR in quotes in dev/check-license.

  rxin

  committed
  95cf203

  Copy full SHA for 95cf203

• More wrapping FWDIR in quotes.

  rxin

  committed
  0feb349

  Copy full SHA for 0feb349

• [SQL] Fix compiling of catalyst docs.

  Show description for 2248891
  marmbrus

  committed
  2248891

  Copy full SHA for 2248891

• dev/check-license wrap folders in quotes.

  rxin

  committed
  437dc8c

  Copy full SHA for 437dc8c

• [SPARK-2024] Add saveAsSequenceFile to PySpark

  Show description for 94d1f46
  

  kanzhang

  authored and

  JoshRosen

  committed
  94d1f46

  Copy full SHA for 94d1f46

• Wrap JAR_DL in dev/check-license.

  rxin

  committed
  7c7ce54

  Copy full SHA for 7c7ce54

• Set AMPLAB_JENKINS_BUILD_PROFILE.

  rxin

  committed
  1097327

  Copy full SHA for 1097327

• Properly pass SBT_MAVEN_PROFILES into sbt.

  rxin

  committed
  2f4b170

  Copy full SHA for 2f4b170

• SPARK-2749 [BUILD]. Spark SQL Java tests aren't compiling in Jenkins' Maven builds; missing junit:junit dep

  Show description for 6ab96a6
  

  srowen

  authored and

  rxin

  committed
  6ab96a6

  Copy full SHA for 6ab96a6

Commits on Jul 31, 2014

• SPARK-2741 - Publish version of spark assembly which does not contain Hive

  Show description for 2ac37db
  

  Brock Noland

  authored and

  pwendell

  committed
  2ac37db

  Copy full SHA for 2ac37db

• [SPARK-2734][SQL] Remove tables from cache when DROP TABLE is run.

  Show description for 88a519d
  marmbrus

  committed
  88a519d

  Copy full SHA for 88a519d

• SPARK-2341 [MLLIB] loadLibSVMFile doesn't handle regression datasets

  Show description for e9b275b
  

  srowen

  authored and

  mengxr

  committed
  e9b275b

  Copy full SHA for e9b275b

• Update DecisionTreeRunner.scala

  Show description for da50176
  

  strat0sphere

  authored and

  mengxr

  committed
  da50176

  Copy full SHA for da50176

• SPARK-2045 Sort-based shuffle

  Show description for e966284
  

  mateiz

  authored and

  rxin

  committed
  e966284

  Copy full SHA for e966284

• [SPARK-2758] UnionRDD's UnionPartition should not reference parent RDDs

  Show description for 894d48f
  rxin

  committed
  894d48f

  Copy full SHA for 894d48f

• Required AM memory is "amMem", not "args.amMemory"

  Show description for 118c1c4
  

  maji2014

  authored and

  pwendell

  committed
  118c1c4

  Copy full SHA for 118c1c4

• [SPARK-2340] Resolve event logging and History Server paths properly

  Show description for a7c305b
  

  andrewor14

  authored and

  pwendell

  committed
  a7c305b

  Copy full SHA for a7c305b

• [SPARK-2737] Add retag() method for changing RDDs' ClassTags.

  Show description for 4fb2593
  JoshRosen

  committed
  4fb2593

  Copy full SHA for 4fb2593

• [SPARK-2497] Included checks for module symbols too.

  Show description for 5a110da
  

  ScrapCodes

  authored and

  pwendell

  committed
  5a110da

  Copy full SHA for 5a110da

• automatically set master according to spark.master in `spark-defaults....

  Show description for 669e3f0
  

  CrazyJvm

  authored and

  pwendell

  committed
  669e3f0

  Copy full SHA for 669e3f0

• [SPARK-2762] SparkILoop leaks memory in multi-repl configurations

  Show description for 92ca910
  

  thunterdb

  authored and

  mateiz

  committed
  92ca910

  Copy full SHA for 92ca910

• [SPARK-2743][SQL] Resolve original attributes in ParquetTableScan

  Show description for 3072b96
  marmbrus

  committed
  3072b96

  Copy full SHA for 3072b96

• [SPARK-2397][SQL] Deprecate LocalHiveContext

  Show description for 72cfb13
  marmbrus

  committed
  72cfb13

  Copy full SHA for 72cfb13

• SPARK-2028: Expose mapPartitionsWithInputSplit in HadoopRDD

  Show description for f193312
  

  aarondav

  authored and

  mateiz

  committed
  f193312

  Copy full SHA for f193312

• SPARK-2664. Deal with --conf options in spark-submit that relate to fl...

  Show description for f68105d
  

  sryza

  authored and

  pwendell

  committed
  f68105d

  Copy full SHA for f68105d

• SPARK-2749 [BUILD] Part 2. Fix a follow-on scalastyle error

  Show description for 4dbabb3
  

  srowen

  authored and

  pwendell

  committed
  4dbabb3

  Copy full SHA for 4dbabb3

• SPARK-2646. log4j initialization not quite compatible with log4j 2.x

  Show description for e5749a1
  

  srowen

  authored and

  pwendell

  committed
  e5749a1

  Copy full SHA for e5749a1

• [SPARK-2511][MLLIB] add HashingTF and IDF

  Show description for dc0865b
  mengxr

  committed
  dc0865b

  Copy full SHA for dc0865b

• [SPARK-2523] [SQL] Hadoop table scan bug fixing (fix failing Jenkins maven test)

  Show description for 49b3612
  

  yhuai

  authored and

  marmbrus

  committed
  49b3612

  Copy full SHA for 49b3612

• Improvements to merge_spark_pr.py

  Show description for e021362
  JoshRosen

  committed
  e021362

  Copy full SHA for e021362

• Docs: monitoring, streaming programming guide

  Show description for cc82050
  

  kennyballou

  authored and

  JoshRosen

  committed
  cc82050

  Copy full SHA for cc82050

• SPARK-2740: allow user to specify ascending and numPartitions for sortBy...

  Show description for 492a195
  

  Rui Li

  authored and

  JoshRosen

  committed
  492a195

  Copy full SHA for 492a195

• SPARK-2282: Reuse Socket for sending accumulator updates to Pyspark

  Show description for ef4ff00
  

  aarondav

  authored and

  JoshRosen

  committed
  ef4ff00

  Copy full SHA for ef4ff00

Commits on Aug 1, 2014

• [SPARK-2531 & SPARK-2436] [SQL] Optimize the BuildSide when planning BroadcastNestedLoopJoin.

  Show description for 8f51491
  

  concretevitamin

  authored and

  marmbrus

  committed
  8f51491

  Copy full SHA for 8f51491

• [SPARK-2724] Python version of RandomRDDGenerators

  Show description for d843014
  

  dorx

  authored and

  mengxr

  committed
  d843014

  Copy full SHA for d843014

• [SPARK-2756] [mllib] Decision tree bug fixes

  Show description for b124de5
  

  jkbradley

  authored and

  mengxr

  committed
  b124de5

  Copy full SHA for b124de5

• [SPARK-2779] [SQL] asInstanceOf[Map[...]] should use scala.collection.Map instead of scala.collection.immutable.Map

  Show description for 9632719
  

  yhuai

  authored and

  marmbrus

  committed
  9632719

  Copy full SHA for 9632719

• SPARK-2766: ScalaReflectionSuite throw an llegalArgumentException in JDK 6

  Show description for 9998efa
  

  witgo

  authored and

  pwendell

  committed
  9998efa

  Copy full SHA for 9998efa

• [SPARK-2777][MLLIB] change ALS factors storage level to MEMORY_AND_DISK

  Show description for b190083
  mengxr

  committed
  b190083

  Copy full SHA for b190083

• [SPARK-2782][mllib] Bug fix for getRanks in SpearmanCorrelation

  Show description for c475540
  

  dorx

  authored and

  mengxr

  committed
  c475540

  Copy full SHA for c475540

• [SPARK-2702][Core] Upgrade Tachyon dependency to 0.5.0

  Show description for 2cdc3e5
  

  haoyuan

  authored and

  pwendell

  committed
  2cdc3e5

  Copy full SHA for 2cdc3e5

• SPARK-2632, SPARK-2576. Fixed by only importing what is necessary during class definition.

  Show description for 1499101
  

  ScrapCodes

  authored and

  marmbrus

  committed
  1499101

  Copy full SHA for 1499101

• SPARK-2738. Remove redundant imports in BlockManagerSuite

  Show description for cb9e7d5
  

  sryza

  authored and

  pwendell

  committed
  cb9e7d5

  Copy full SHA for cb9e7d5

• [SPARK-2670] FetchFailedException should be thrown when local fetch has failed

  Show description for 8ff4417
  

  sarutak

  authored and

  mateiz

  committed
  8ff4417

  Copy full SHA for 8ff4417

• SPARK-983. Support external sorting in sortByKey()

  Show description for 72e3369
  mateiz

  committed
  72e3369

  Copy full SHA for 72e3369

• SPARK-2134: Report metrics before application finishes

  Show description for f1957e1
  

  Rahul Singhal

  authored and

  mateiz

  committed
  f1957e1

  Copy full SHA for f1957e1

• [Spark 2557] fix LOCAL_N_REGEX in createTaskScheduler and make local-n and local-n-failures consistent

  Show description for 284771e
  

  advancedxy

  authored and

  aarondav

  committed
  284771e

  Copy full SHA for 284771e

• [SPARK-2103][Streaming] Change to ClassTag for KafkaInputDStream and fix reflection issue

  Show description for a32f0fb
  

  jerryshao

  authored and

  tdas

  committed
  a32f0fb

  Copy full SHA for a32f0fb

• SPARK-2768 [MLLIB] Add product, user recommend method to MatrixFactorizationModel

  Show description for 82d209d
  

  srowen

  authored and

  mengxr

  committed
  82d209d

  Copy full SHA for 82d209d

• [SPARK-1997] update breeze to version 0.8.1

  Show description for 0dacb1a
  

  witgo

  authored and

  mengxr

  committed
  0dacb1a

  Copy full SHA for 0dacb1a

• [HOTFIX] downgrade breeze version to 0.7

  Show description for 5328c0a
  mengxr

  committed
  5328c0a

  Copy full SHA for 5328c0a

• SPARK-2099. Report progress while task is running.

  Show description for 8d338f6
  

  sryza

  authored and

  pwendell

  committed
  8d338f6

  Copy full SHA for 8d338f6

• [SPARK-2179][SQL] A minor refactoring Java data type APIs (2179 follow-up).

  Show description for c41fdf0
  

  yhuai

  authored and

  marmbrus

  committed
  c41fdf0

  Copy full SHA for c41fdf0

• [SQL][SPARK-2212]Hash Outer Join

  Show description for 4415722
  

  chenghao-intel

  authored and

  marmbrus

  committed
  4415722

  Copy full SHA for 4415722

• [SPARK-2729] [SQL] Forgot to match Timestamp type in ColumnBuilder

  Show description for 580c701
  

  chutium

  authored and

  marmbrus

  committed
  580c701

  Copy full SHA for 580c701

• [SPARK-2767] [SQL] SparkSQL CLI doens't output error message if query failed.

  Show description for c0b47ba
  

  chenghao-intel

  authored and

  marmbrus

  committed
  c0b47ba

  Copy full SHA for c0b47ba

• [SQL] Documentation: Explain cacheTable command

  Show description for c82fe47
  

  CrazyJvm

  authored and

  pwendell

  committed
  c82fe47

  Copy full SHA for c82fe47

• [SPARK-695] In DAGScheduler's getPreferredLocs, track set of visited partitions.

  Show description for eb5bdca
  

  staple

  authored and

  mateiz

  committed
  eb5bdca

  Copy full SHA for eb5bdca

• [SPARK-2490] Change recursive visiting on RDD dependencies to iterative approach

  Show description for baf9ce1
  

  viirya

  authored and

  mateiz

  committed
  baf9ce1

  Copy full SHA for baf9ce1

• SPARK-1612: Fix potential resource leaks

  Show description for f5d9bea
  

  zsxwing

  authored and

  mateiz

  committed
  f5d9bea

  Copy full SHA for f5d9bea

• [SPARK-2379] Fix the bug that streaming's receiver may fall into a dead loop

  Show description for b270309
  

  joyyoj

  authored and

  tdas

  committed
  b270309

  Copy full SHA for b270309

• SPARK-2791: Fix committing, reverting and state tracking in shuffle file consolidation

  Show description for 78f2af5
  

  aarondav

  authored and

  mateiz

  committed
  78f2af5

  Copy full SHA for 78f2af5

• [SPARK-2786][mllib] Python correlations

  Show description for d88e695
  

  dorx

  authored and

  mengxr

  committed
  d88e695

  Copy full SHA for d88e695

• [SPARK-2796] [mllib] DecisionTree bug fix: ordered categorical features

  Show description for 7058a53
  

  jkbradley

  authored and

  mengxr

  committed
  7058a53

  Copy full SHA for 7058a53

Commits on Aug 2, 2014

• [SPARK-2010] [PySpark] [SQL] support nested structure in SchemaRDD

  Show description for 880eabe
  

  davies

  authored and

  marmbrus

  committed
  880eabe

  Copy full SHA for 880eabe

• [SPARK-2212][SQL] Hash Outer Join (follow-up bug fix).

  Show description for 3822f33
  

  yhuai

  authored and

  marmbrus

  committed
  3822f33

  Copy full SHA for 3822f33

• [SPARK-2116] Load spark-defaults.conf from SPARK_CONF_DIR if set

  Show description for 0da07da
  

  chu11

  authored and

  mateiz

  committed
  0da07da

  Copy full SHA for 0da07da

• [SPARK-2800]: Exclude scalastyle-output.xml Apache RAT checks

  Show description for a38d3c9
  

  witgo

  authored and

  pwendell

  committed
  a38d3c9

  Copy full SHA for a38d3c9

• [SPARK-2764] Simplify daemon.py process structure

  Show description for e8e0fd6
  

  JoshRosen

  authored and

  aarondav

  committed
  e8e0fd6

  Copy full SHA for e8e0fd6

• Streaming mllib [SPARK-2438][MLLIB]

  Show description for f6a1899
  

  freeman-lab

  authored and

  mengxr

  committed
  f6a1899

  Copy full SHA for f6a1899

• [SPARK-2550][MLLIB][APACHE SPARK] Support regularization and intercept in pyspark's linear methods.

  Show description for c281189
  

  miccagiann

  authored and

  mengxr

  committed
  c281189

  Copy full SHA for c281189

• [SPARK-1580][MLLIB] Estimate ALS communication and computation costs.

  Show description for e25ec06
  

  tmyklebu

  authored and

  mengxr

  committed
  e25ec06

  Copy full SHA for e25ec06

• [SPARK-2801][MLlib]: DistributionGenerator renamed to RandomDataGenerator. RandomRDD is now of generic type

  Show description for fda4759
  

  brkyvz

  authored and

  mengxr

  committed
  fda4759

  Copy full SHA for fda4759

• StatCounter on NumPy arrays [PYSPARK][SPARK-2012]

  Show description for 4bc3bb2
  

  freeman-lab

  authored and

  JoshRosen

  committed
  4bc3bb2

  Copy full SHA for 4bc3bb2

• [SPARK-1470][SPARK-1842] Use the scala-logging wrapper instead of the directly sfl4j api

  Show description for adc8303
  

  witgo

  authored and

  pwendell

  committed
  adc8303

  Copy full SHA for adc8303

• Revert "[SPARK-1470][SPARK-1842] Use the scala-logging wrapper instead of the directly sfl4j api"

  Show description for dab3796
  pwendell

  committed
  dab3796

  Copy full SHA for dab3796

• [SPARK-2316] Avoid O(blocks) operations in listeners

  Show description for d934801
  

  andrewor14

  authored and

  pwendell

  committed
  d934801

  Copy full SHA for d934801

• [SPARK-2454] Do not ship spark home to Workers

  Show description for 148af60
  

  andrewor14

  authored and

  pwendell

  committed
  148af60

  Copy full SHA for 148af60

• [SPARK-1812] sql/catalyst - Provide explicit type information

  Show description for 08c095b
  

  avati

  authored and

  marmbrus

  committed
  08c095b

  Copy full SHA for 08c095b

• HOTFIX: Fixing test error in maven for flume-sink.

  Show description for 25cad6a
  pwendell

  committed
  25cad6a

  Copy full SHA for 25cad6a

• HOTFIX: Fix concurrency issue in FlumePollingStreamSuite.

  Show description for 44460ba
  pwendell

  committed
  44460ba

  Copy full SHA for 44460ba

• MAINTENANCE: Automated closing of pull requests.

  Show description for 87738bf
  pwendell

  committed
  87738bf

  Copy full SHA for 87738bf

• [HOTFIX] Do not throw NPE if spark.test.home is not set

  Show description for e09e18b
  

  andrewor14

  authored and

  pwendell

  committed
  e09e18b

  Copy full SHA for e09e18b

• [SPARK-2478] [mllib] DecisionTree Python API

  Show description for 3f67382
  

  jkbradley

  authored and

  mengxr

  committed
  3f67382

  Copy full SHA for 3f67382

• [SQL] Set outputPartitioning of BroadcastHashJoin correctly.

  Show description for 67bd8e3
  

  yhuai

  authored and

  marmbrus

  committed
  67bd8e3

  Copy full SHA for 67bd8e3

• [SPARK-1981] Add AWS Kinesis streaming support

  Show description for 91f9504
  

  cfregly

  authored and

  tdas

  committed
  91f9504

  Copy full SHA for 91f9504

• SPARK-2804: Remove scalalogging-slf4j dependency

  Show description for 4c47711
  

  witgo

  authored and

  pwendell

  committed
  4c47711

  Copy full SHA for 4c47711

• [SPARK-2097][SQL] UDF Support

  Show description for 158ad0b
  marmbrus

  committed
  158ad0b

  Copy full SHA for 158ad0b

• [SPARK-2785][SQL] Remove assertions that throw when users try unsupported Hive commands.

  Show description for 198df11
  marmbrus

  committed
  198df11

  Copy full SHA for 198df11

Commits on Aug 3, 2014

• [SPARK-2729][SQL] Added test case for SPARK-2729

  Show description for 866cf1f
  

  liancheng

  authored and

  marmbrus

  committed
  866cf1f

  Copy full SHA for 866cf1f

• [SPARK-2797] [SQL] SchemaRDDs don't support unpersist()

  Show description for d210022
  

  yhuai

  authored and

  marmbrus

  committed
  d210022

  Copy full SHA for d210022

• [SPARK-2739][SQL] Rename registerAsTable to registerTempTable

  Show description for 1a80437
  marmbrus

  committed
  1a80437

  Copy full SHA for 1a80437

• SPARK-2602 [BUILD] Tests steal focus under Java 6

  Show description for 33f167d
  

  srowen

  authored and

  pwendell

  committed
  33f167d

  Copy full SHA for 33f167d

• SPARK-2414 [BUILD] Add LICENSE entry for jquery

  Show description for 9cf429a
  

  srowen

  authored and

  pwendell

  committed
  9cf429a

  Copy full SHA for 9cf429a

• [Minor] Fixes on top of #1679

  Show description for 3dc55fd
  

  andrewor14

  authored and

  pwendell

  committed
  3dc55fd

  Copy full SHA for 3dc55fd

• SPARK-2712 - Add a small note to maven doc that mvn package must happen ...

  Show description for f8cd143
  

  javadba

  authored and

  pwendell

  committed
  f8cd143

  Copy full SHA for f8cd143

• SPARK-2246: Add user-data option to EC2 scripts

  Show description for a0bcbc1
  

  Allan Douglas R. de Oliveira

  authored and

  pwendell

  committed
  a0bcbc1

  Copy full SHA for a0bcbc1

• [SPARK-2197] [mllib] Java DecisionTree bug fix and easy-of-use

  Show description for 2998e38
  

  jkbradley

  authored and

  mengxr

  committed
  2998e38

  Copy full SHA for 2998e38

• [SPARK-2784][SQL] Deprecate hql() method in favor of a config option, 'spark.sql.dialect'

  Show description for 236dfac
  marmbrus

  committed
  236dfac

  Copy full SHA for 236dfac

• [SPARK-2814][SQL] HiveThriftServer2 throws NPE when executing native commands

  Show description for ac33cbb
  

  liancheng

  authored and

  marmbrus

  committed
  ac33cbb

  Copy full SHA for ac33cbb

• [SPARK-2783][SQL] Basic support for analyze in HiveContext

  Show description for e139e2b
  

  yhuai

  authored and

  marmbrus

  committed
  e139e2b

  Copy full SHA for e139e2b

• [SPARK-1740] [PySpark] kill the python worker

  Show description for 55349f9
  

  davies

  authored and

  JoshRosen

  committed
  55349f9

  Copy full SHA for 55349f9

Commits on Aug 4, 2014

• [SPARK-2810] upgrade to scala-maven-plugin 3.2.0

  Show description for 6ba6c3e
  

  avati

  authored and

  pwendell

  committed
  6ba6c3e

  Copy full SHA for 6ba6c3e

• Fix some bugs with spaces in directory name.

  Show description for 5507dd8
  

  sarahgerweck

  authored and

  pwendell

  committed
  5507dd8

  Copy full SHA for 5507dd8

• SPARK-2272 [MLlib] Feature scaling which standardizes the range of independent variables or features of data

  Show description for ae58aea
  

  DB Tsai

  authored and

  mengxr

  committed
  ae58aea

  Copy full SHA for ae58aea

• [MLlib] [SPARK-2510]Word2Vec: Distributed Representation of Words

  Show description for e053c55
  

  Liquan Pei

  authored and

  mengxr

  committed
  e053c55

  Copy full SHA for e053c55

• [SPARK-1687] [PySpark] pickable namedtuple

  Show description for 59f84a9
  

  davies

  authored and

  JoshRosen

  committed
  59f84a9

  Copy full SHA for 59f84a9

• SPARK-2792. Fix reading too much or too little data from each stream in ExternalMap / Sorter

  Show description for 8e7d5ba
  mateiz

  committed
  8e7d5ba

  Copy full SHA for 8e7d5ba

• [SPARK-1687] [PySpark] fix unit tests related to pickable namedtuple

  Show description for 9fd82db
  

  davies

  authored and

  JoshRosen

  committed
  9fd82db

  Copy full SHA for 9fd82db

Commits on Aug 5, 2014

• [SPARK-2323] Exception in accumulator update should not crash DAGScheduler & SparkContext

  Show description for 05bf4e4
  rxin

  committed
  05bf4e4

  Copy full SHA for 05bf4e4

• SPARK-2685. Update ExternalAppendOnlyMap to avoid buffer.remove()

  Show description for 066765d
  mateiz

  committed
  066765d

  Copy full SHA for 066765d

• SPARK-2711. Create a ShuffleMemoryManager to track memory for all spilling collections

  Show description for 4fde28c
  mateiz

  committed
  4fde28c

  Copy full SHA for 4fde28c

• [SPARK-2857] Correct properties to set Master / Worker ports

  Show description for a646a36
  

  andrewor14

  authored and

  pwendell

  committed
  a646a36

  Copy full SHA for a646a36

• [SPARK-1779] Throw an exception if memory fractions are not between 0 and 1

  Show description for 9862c61
  

  wangfei

  authored and

  pwendell

  committed
  9862c61

  Copy full SHA for 9862c61

• [SPARK-2856] Decrease initial buffer size for Kryo to 64KB.

  Show description for 184048f
  rxin

  committed
  184048f

  Copy full SHA for 184048f

• [SPARK-1022][Streaming] Add Kafka real unit test

  Show description for e87075d
  

  jerryshao

  authored and

  tdas

  committed
  e87075d

  Copy full SHA for e87075d

• SPARK-1528 - spark on yarn, add support for accessing remote HDFS

  Show description for 2c0f705
  tgravescs

  committed
  2c0f705

  Copy full SHA for 2c0f705

• SPARK-1890 and SPARK-1891- add admin and modify acls

  Show description for 1c5555a
  tgravescs

  committed
  1c5555a

  Copy full SHA for 1c5555a

• [SPARK-2860][SQL] Fix coercion of CASE WHEN.

  Show description for 6e821e3
  marmbrus

  committed
  6e821e3

  Copy full SHA for 6e821e3

• [SPARK-2859] Update url of Kryo project in related docs

  Show description for ac3440f
  

  gchen

  authored and

  pwendell

  committed
  ac3440f

  Copy full SHA for ac3440f

• SPARK-2380: Support displaying accumulator values in the web UI

  Show description for 74f82c7
  pwendell

  committed
  74f82c7

  Copy full SHA for 74f82c7

• SPARK-1680: use configs for specifying environment variables on YARN

  Show description for 41e0a21
  tgravescs

  committed
  41e0a21

  Copy full SHA for 41e0a21

• [SPARK-2864][MLLIB] fix random seed in word2vec; move model to local

  Show description for cc491f6
  mengxr

  committed
  cc491f6

  Copy full SHA for cc491f6

• [SPARK-2503] Lower shuffle output buffer (spark.shuffle.file.buffer.kb) to 32KB.

  Show description for acff9a7
  rxin

  committed
  acff9a7

  Copy full SHA for acff9a7

• [SPARK-2550][MLLIB][APACHE SPARK] Support regularization and intercept in pyspark's linear methods

  Show description for 1aad911
  

  miccagiann

  authored and

  mengxr

  committed
  1aad911

  Copy full SHA for 1aad911

Commits on Aug 6, 2014

• SPARK-2869 - Fix tiny bug in JdbcRdd for closing jdbc connection

  Show description for 2643e66
  

  Stephen Boesch

  authored and

  rxin

  committed
  2643e66

  Copy full SHA for 2643e66

• [sql] rename project name in pom.xml of hive-thriftserver module

  Show description for d94f599
  

  scwf

  authored and

  marmbrus

  committed
  d94f599

  Copy full SHA for d94f599

• [SPARK-2650][SQL] Try to partially fix SPARK-2650 by adjusting initial buffer size and reducing memory allocation

  Show description for d0ae3f3
  

  liancheng

  authored and

  marmbrus

  committed
  d0ae3f3

  Copy full SHA for d0ae3f3

• [SPARK-2854][SQL] Finalize _acceptable_types in pyspark.sql

  Show description for 69ec678
  

  yhuai

  authored and

  marmbrus

  committed
  69ec678

  Copy full SHA for 69ec678

• [SPARK-2866][SQL] Support attributes in ORDER BY that aren't in SELECT

  Show description for 1d70c4f
  marmbrus

  committed
  1d70c4f

  Copy full SHA for 1d70c4f

• [SPARK-2806] core - upgrade to json4s-jackson 3.2.10

  Show description for 82624e2
  

  avati

  authored and

  pwendell

  committed
  82624e2

  Copy full SHA for 82624e2

• [SQL] Tighten the visibility of various SQLConf methods and renamed setter/getters

  Show description for b70bae4
  

  rxin

  authored and

  marmbrus

  committed
  b70bae4

  Copy full SHA for b70bae4

• [SQL] Fix logging warn -> debug

  Show description for 5a826c0
  marmbrus

  committed
  5a826c0

  Copy full SHA for 5a826c0

• SPARK-2294: fix locality inversion bug in TaskManager

  Show description for 63bdb1f
  

  CodingCat

  authored and

  mateiz

  committed
  63bdb1f

  Copy full SHA for 63bdb1f

• [MLlib] Use this.type as return type in k-means' builder pattern

  Show description for c7b5201
  

  DB Tsai

  authored and

  mengxr

  committed
  c7b5201

  Copy full SHA for c7b5201

• [SPARK-1022][Streaming][HOTFIX] Fixed zookeeper dependency of Kafka

  Show description for ee7f308
  

  tdas

  authored and

  pwendell

  committed
  ee7f308

  Copy full SHA for ee7f308

• [SPARK-2157] Enable tight firewall rules for Spark

  Show description for 09f7e45
  

  andrewor14

  authored and

  pwendell

  committed
  09f7e45

  Copy full SHA for 09f7e45