python-swat icon indicating copy to clipboard operation
python-swat copied to clipboard

applying formats using fetch or to_frame makes all DataFrame columns objects

Open pestyld opened this issue 2 years ago • 0 comments
trafficstars

If you have formats on only a few columns in a CAS table and try to apply the formats to the SASDataFrame, the swat package converts all columns to objects, even if the column is a simple number.

For example:

Packages

import swat import pandas as pd

Connect to CAS

conn = ##connection info

load data to CAS

data = r'https://support.sas.com/documentation/onlinedoc/viya/exampledatasets/cars.csv' tbl = conn.upload_file(data, casout = {'name':'cars_upload', 'caslib':'casuser','replace':True}, importoptions={ 'stripBlanks':True, 'guessRows':200, 'vars':{ 'MSRP':{'format':'dollar16.'}, 'Weight':{'format':'comma16.'} }})

Only the MSRP and Weight columns have a format

tbl.columnInfo()

image

Pull the CAS table to the client as a SASDataFrame and apply the formats. This works as expected

df = tbl.to_frame(format=True) df.head()

image

View the data types of the SASDataFrame

image

Everything turns into an object. It seems like only the formatted columns should be converted. Numeric columns without formats should remain numbers.

pestyld avatar Dec 22 '22 13:12 pestyld