Details
-
Bug
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
None
-
None
Description
Livy SQL Interpreter uses df.show( n ) which truncates string columns in result if they are more than 20 characters long.
http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.sql.Dataset@show(numRows:Int,truncate:Boolean):Unit
http://spark.apache.org/docs/latest/api/python/pyspark.sql.html#pyspark.sql.DataFrame.show
df.show(1000, false) or df.show(1000, 1000) should be used to display results unchanged or truncated when strings are actually too long to display.
Other option is to make it configurable. Possible values are true, false, integer > 1
Attachments
Issue Links
- links to