How to ADD New Columns in DataFrame using PySpark

 Below are different ways to add new columns to dataframe in PySpark:

1. withColumn and lit
1. df.withColumn("NewColumnName", lit("default value for new column"))
2. withColumn and col (Derived column)
1. df.withColumn("NewColumnName", col("Column1") * col("Column2"))
3. select
1. df.select(lit("default column value").alias("NewColumnName"), col("Column1"), col("Column2"))

How to Rename columns in DataFrame using PySpark

 There are multiple ways to rename columns in dataframe using PySpark.

1. withColumnRenamed
1. df = df.withColumnRenamed("Old_ColumnName1", "New_ColumnName1").withColumnRenamed("Old_ColumnName2", "New_ColumnName2")
2. selectExpr
1. df = df.selectExpr("Old_ColumnName1 AS NewColumnName1","Old_ColumnName2 AS NewColumnName2")
3. select(col().alias(), col())
1. df2 = df.select(col("Old_ColumnName1").alias("NewColumnName1"), col("Old_ColumnName2"))