Spark sql function

[liu0013]

@liancheng I want to consult a question about spark sql function: from_utc_timestamp(ts: Column, tz: String) I am using mongo-spark to load "member" collection from mongodb which is included three fields: memberId, date, timezone. case class Member(memberId: String, date: Timestamp, timezone: String) val memberDF: Dataframe = load [ Member ] ("member") I want to invoke from_utc_timestamp to get member's timezone timestamp, memberDF.select(memberId, from_utc_timestamp(date, timezone)), however, tz type is String, it is not a column type. how to implement from_utc_timestamp(ts:Column, tz:Column)?

def from_utc_timestamp(ts: Column, tz: String): Column = withExpr { FromUTCTimestamp(ts.expr, Literal(tz)) }

withExpr is private method......

Thanks, Aaron

[Seidzi]

Hello! The timestamp always has UTC+0, You need the member's location to deduct the time zone

[liu0013]

Thanks @Seidzi ! I want to create daily report by local date, so if from_utc_timestamp(ts:Column, tz:Column) is implemented then I can get the local date by timezone: to_date(from_utc_timestamp(current_timestamp, timezoneId))