This yields the same output as previous example. Spark.sql('select Seqno, convertUDF(Quote) from QUOTE_TABLE').show(false) ('convertUDF', convertCase)Дf.createOrReplaceTempView('QUOTE_TABLE') In order to use convertCase() function on Spark SQL, you need to register the function with Spark using (). |3 |The Purpose Of Our Lives Is To Be Happy. |2 |Everyone Thinks Of Changing The World, But No One Thinks Of Changing Himself.| |1 |Be The Change That You Wish To See In The World | udf() function return .expressions.UserDefinedFunction.Д¬onvertUDF(col('Quote')).as('Quote') ).show(false) Now you can use convertUDF() on a DataFrame column. Make sure you import this package before using it. Now convert this function convertCase() to UDF by passing the function to Spark SQL udf(), this function is available at .functions.udf package. UDFвЂ™s take parameters of your choice and returns a value.ДЄrr.map(f=> f.substring(0,1).toUpperCase + f.substring(1,f.length)).mkString(' ') Below snippet creates a function convertCase() which takes a string parameter and converts the first letter of every word to capital letter. The first step in creating a UDF is creating a Scala function. |3 |The purpose of our lives is to be happy. |2 |Everyone thinks of changing the world, but no one thinks of changing himself.| |1 |Be the change that you wish to see in the world | ('3', 'The purpose of our lives is to be happy.') ('2', 'Everyone thinks of changing the world, but no one thinks of changing himself.'), Val data = Seq(('1', 'Be the change that you wish to see in the world'),

Instantburn udf driver download