spark auto infer col(all string) types

if we have many columns in one table, and all the columns are String. The problem is how to infer the real types. sample_json = ''' { "a": "123", "b": true, "c": "2022-01-01T18:00:00+8:00[Asia/Shanghai]" } ''' # provide the sample json and infer types by it schemas = schema_of_json(sample_json) spark = ... df = ... df.select(from_json(payload_json, schemas).alias("data")).select("data.*") .withColumn("created_time", to_timestamp(regexp_replace(col("a"), r'\.\d+|\[.*\]', ''), "yyyy-MM-dd'T'HH:mm:ssXXX")) ...

spark many columns to rows

exist table: id name attr1 attr2 attr3 attr4 attr5 attr6 attr7 attr8 1 aaa true true true true true true false false 2 bbb false false false false true false false false 3 ccc true false false false false true false false target table: id name attr flag 1 aaa attr1 true 1 aaa attr2 true 1 aaa attr3 true 1 aaa attr4 true … … … … 2 bbb attr1 false 2 bbb attr2 false … … … … python spark:

Linux SMB2 share

Commands: sudo addgroup smbgrp sudo useradd shares -G smbgrp sudo smbpasswd -a shares sudo mkdir -p /srv/samba-secured sudo chmod -R 0770 /srv/samba-secured sudo chown root:smbgrp /srv/samba-secured Configs( /etc/samba/smb.conf ): [global] # protocol version range, currently only SMB2 min protocol = SMB2 max protocol = SMB2 [SECURED] path = /srv/samba-secured valid users = @smbgrp browseable = yes writable = yes read only = no restart it: sudo service smbd restart

AttributeGraph: cycle detected

When you are going to apply any value changes, you need to close the keyboard first: func closeKeyboard() { UIApplication.shared.sendAction( #selector(UIResponder.resignFirstResponder), to: nil, from: nil, for: nil ) }