Convert CSV to Out object

[lrocax]

Hi,

Great package.

A question: I have a CSV that contain 4 columns (objectName,Date,Location,Coord) with several rows. How can I import the objectName and Coord as Out object?

Ex1.zip

Thanks

[vbonhomme]

Where does it come from and what have you tried ?

Le dim. 17 mai 2020 17:20, Lluis Roca [email protected] a écrit :

Hi,

Great package.

A question: I have a CSV that contain 4 columns (objectName,Date,Location,Coord) with several rows. How can I import the objectName and Coord as Out object?

Ex1.zip https://github.com/MomX/Momocs/files/4640478/Ex1.zip

Thanks

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/MomX/Momocs/issues/211, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABECQFWPIPFCLZYDIW2KS43RR76CZANCNFSM4NDNWFNA .

[vbonhomme]

I now have a real keyboard ! The where does it come from is because of Momit, I'm trying to cover all possible sources for raw data. The what have you tried is because I believe in you !

Le dim. 17 mai 2020 à 19:53, Vincent Bonhomme [email protected] a écrit :

Where does it come from and what have you tried ?

Le dim. 17 mai 2020 17:20, Lluis Roca [email protected] a écrit :

Hi,

Great package.

A question: I have a CSV that contain 4 columns (objectName,Date,Location,Coord) with several rows. How can I import the objectName and Coord as Out object?

Ex1.zip https://github.com/MomX/Momocs/files/4640478/Ex1.zip

Thanks

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/MomX/Momocs/issues/211, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABECQFWPIPFCLZYDIW2KS43RR76CZANCNFSM4NDNWFNA .

-- vincentbonhomme.fr http://www.vincentbonhomme.fr

[lrocax]

It is an output from a python code that I wrote that is structured as followed [x1 y1; x2 y2; …xn yn]. So, a general solution for CSV file will be awesome.

[petrpajdla]

Generally speaking, the structure you showed in your example is not a simple CSV file, it resembles file structures used with spatial data. I think you first need to read the coordinates and add the metadata to an object with coordinates in the next step...

[vbonhomme]

Hey there,

I have got a bitt of code that does that (see below). Apparently this type of structure is not rare so I ll verse it into Momit. Try to survive with this, if you dont ring my bell ! Read your .csv first so that it enters the function as a data.frame.

# Dependencies
library(tidyverse)
library(Momocs)

# a function that turns the csv with coordinates
# encoded into columns into a proper Out with its fac
from_csv_to_Out <- function(df){
  df <- as_tibble(df) # cosmetics
  fac <- df %>% select(-matches("X[[:digit:]]+(x|y)"))     # all but non
coord columns -> our $fac
  coo <- df %>% select(matches("X[[:digit:]]+(x|y)")) %>%  # all coord
columns         -> our $coo
    #split rowwise
    split(1:nrow(.)) %>%
    # for each row, turn columns into shape
    map(~.x %>%
          # select appropriate columns
          select(matches("X[[:digit:]]+(x|y)")) %>%
          # unlist and turn into a shape
          unlist() %>%
          na.omit() %>% # required because some missing data
          matrix(ncol=2, byrow=T))
  # build  and return the Out
  Out(coo, fac)
}

Le lun. 18 mai 2020 à 09:01, Petr Pajdla [email protected] a écrit :

Generally speaking, the structure you showed in your example is not a simple CSV file, it resembles file structures used with spatial data. I think you first need to read the coordinates and add the metadata to an object with coordinates in the next step...

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MomX/Momocs/issues/211#issuecomment-629985991, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABECQFWX3BCF7EFY5NX5YULRSDMMPANCNFSM4NDNWFNA .

-- vincentbonhomme.fr http://www.vincentbonhomme.fr

[raz1]

I was running this code using R 4.0.5

library(tidyverse)
library(Momocs)
 
df <- read.csv(file = 'E:/Ex1.csv')
  
from_csv_to_Out <- function(df){
  df <- as_tibble(df) # cosmetics
  fac <- df %>% select(-matches("X[[:digit:]]+(x|y)"))     # all but non
  coord columns -> our $fac
  coo <- df %>% select(matches("X[[:digit:]]+(x|y)")) %>%  # all coord
    columns         -> our $coo
  #split rowwise
  split(1:nrow(.)) %>%
    # for each row, turn columns into shape
    map(~.x %>%
          # select appropriate columns
          select(matches("X[[:digit:]]+(x|y)")) %>%
          # unlist and turn into a shape
          unlist() %>%
          na.omit() %>% # required because some missing data
          matrix(ncol=2, byrow=T))
  # build  and return the Out
  Out(coo, fac)
}

and I got this output:

library(tidyverse)
-- Attaching packages ------------------------------------------ tidyverse 1.3.1 --
<U+221A> ggplot2 3.3.4     <U+221A> purrr   0.3.4
<U+221A> tibble  3.1.2     <U+221A> dplyr   1.0.6
<U+221A> tidyr   1.1.3     <U+221A> stringr 1.4.0
<U+221A> readr   1.4.0     <U+221A> forcats 0.5.1
-- Conflicts --------------------------------------------- tidyverse_conflicts() --
x dplyr::filter() masks stats::filter()
x dplyr::lag()    masks stats::lag()
> library(Momocs)

Attaching package: ‘Momocs’

The following objects are masked from ‘package:dplyr’:

   arrange, combine, filter, mutate, rename, sample_frac, sample_n,
   select, slice

The following object is masked from ‘package:tidyr’:

   chop

The following object is masked from ‘package:stats’:

   filter

> df <- read.csv(file = 'E:/Ex1.csv')
> from_csv_to_Out <- function(df){
+   df <- as_tibble(df) # cosmetics
+   fac <- df %>% select(-matches("X[[:digit:]]+(x|y)"))     # all but non
+   coord columns -> our $fac
Error: unexpected symbol in:
"  fac <- df %>% select(-matches("X[[:digit:]]+(x|y)"))     # all but non
 coord columns"
>   coo <- df %>% select(matches("X[[:digit:]]+(x|y)")) %>%  # all coord
+     columns         -> our $coo
Error in columns(.) : could not find function "columns"
>   #split rowwise
>   split(1:nrow(.)) %>%
+     # for each row, turn columns into shape
+     map(~.x %>%
+           # select appropriate columns
+           select(matches("X[[:digit:]]+(x|y)")) %>%
+           # unlist and turn into a shape
+           unlist() %>%
+           na.omit() %>% # required because some missing data
+           matrix(ncol=2, byrow=T))
Error in nrow(.) : object '.' not found
>   # build  and return the Out
>   Out(coo, fac)
Error in Out(coo, fac) : object 'coo' not found
> }
Error: unexpected '}' in "}"

Any ideas how to solve the problem?

[vbonhomme]

the two line with "->" shoulb be commented, ie:

from_csv_to_Out <- function(df){

  df <- as_tibble(df) # cosmetics

  fac <- df %>% select(-matches("X[[:digit:]]+(x|y)"))     # all but non



  coo <- df %>% select(matches("X[[:digit:]]+(x|y)")) %>%  # all coord



  #split rowwise

  split(1:nrow(.)) %>%

    # for each row, turn columns into shape

    map(~.x %>%

          # select appropriate columns

          select(matches("X[[:digit:]]+(x|y)")) %>%

          # unlist and turn into a shape

          unlist() %>%

          na.omit() %>% # required because some missing data

          matrix(ncol=2, byrow=T))

  # build  and return the Out

  Out(coo, fac)

}

Le mer. 23 juin 2021 à 14:48, Ely Raz @.***> a écrit :

I was running this code using R 4.0.5

library(tidyverse)

library(Momocs)

df <- read.csv(file = 'E:/Ex1.csv')

from_csv_to_Out <- function(df){

df <- as_tibble(df) # cosmetics

fac <- df %>% select(-matches("X[[:digit:]]+(x|y)")) # all but non

coord columns -> our $fac

coo <- df %>% select(matches("X[[:digit:]]+(x|y)")) %>% # all coord

columns         -> our $coo

#split rowwise

split(1:nrow(.)) %>%

# for each row, turn columns into shape

map(~.x %>%

      # select appropriate columns

      select(matches("X[[:digit:]]+(x|y)")) %>%

      # unlist and turn into a shape

      unlist() %>%

      na.omit() %>% # required because some missing data

      matrix(ncol=2, byrow=T))

build and return the Out

Out(coo, fac)

}

and I got this output:

library(tidyverse)

-- Attaching packages ------------------------------------------ tidyverse 1.3.1 --

<U+221A> ggplot2 3.3.4 <U+221A> purrr 0.3.4

<U+221A> tibble 3.1.2 <U+221A> dplyr 1.0.6

<U+221A> tidyr 1.1.3 <U+221A> stringr 1.4.0

<U+221A> readr 1.4.0 <U+221A> forcats 0.5.1

-- Conflicts --------------------------------------------- tidyverse_conflicts() --

x dplyr::filter() masks stats::filter()

x dplyr::lag() masks stats::lag()

library(Momocs)

Attaching package: ‘Momocs’

The following objects are masked from ‘package:dplyr’:

arrange, combine, filter, mutate, rename, sample_frac, sample_n,

select, slice

The following object is masked from ‘package:tidyr’:

chop

The following object is masked from ‘package:stats’:

filter

df <- read.csv(file = 'E:/Ex1.csv')

from_csv_to_Out <- function(df){

• df <- as_tibble(df) # cosmetics

• fac <- df %>% select(-matches("X[[:digit:]]+(x|y)")) # all but non

• coord columns -> our $fac

Error: unexpected symbol in:

" fac <- df %>% select(-matches("X[[:digit:]]+(x|y)")) # all but non

coord columns"

coo <- df %>% select(matches("X[[:digit:]]+(x|y)")) %>% # all coord

• columns         -> our $coo
  

Error in columns(.) : could not find function "columns"

#split rowwise

split(1:nrow(.)) %>%

• # for each row, turn columns into shape
  

• map(~.x %>%
  

•       # select appropriate columns
  

•       select(matches("X[[:digit:]]+(x|y)")) %>%
  

•       # unlist and turn into a shape
  

•       unlist() %>%
  

•       na.omit() %>% # required because some missing data
  

•       matrix(ncol=2, byrow=T))
  

Error in nrow(.) : object '.' not found

build and return the Out

Out(coo, fac)

Error in Out(coo, fac) : object 'coo' not found

}

Error: unexpected '}' in "}"

Any ideas how to solve the problem?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MomX/Momocs/issues/211#issuecomment-866806109, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABECQFS3PLX42EC3E3XCE6TTUHJ3JANCNFSM4NDNWFNA .

-- vincentbonhomme.fr http://www.vincentbonhomme.fr

[raz1]

Thanks for the super fast respond.

I tried this line with this output:

> ex1 <- from_csv_to_Out(df)
 Error in matrix(., ncol = 2, byrow = T) : 
  'data' must be of a vector type, was 'NULL' 

I did something wrong?

[vbonhomme]

from_csv_to_Out expects a filepath I guess

Le mer. 23 juin 2021 à 15:08, Ely Raz @.***> a écrit :

Thanks for the super fast respond.

I tried this line with this output:

ex1 <- from_csv_to_Out(df) Error in matrix(., ncol = 2, byrow = T) : 'data' must be of a vector type, was 'NULL'

I did something wrong?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MomX/Momocs/issues/211#issuecomment-866819870, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABECQFRUILMLLGQEO5AP3O3TUHMDHANCNFSM4NDNWFNA .

-- vincentbonhomme.fr http://www.vincentbonhomme.fr

[raz1]

The same error

> ex1 <- from_csv_to_Out('E:/Ex1.csv')
 Error in matrix(., ncol = 2, byrow = T) : 
  'data' must be of a vector type, was 'NULL' 

[vbonhomme]

send me "Ex1.csv" > @.***

Le mer. 23 juin 2021 à 15:13, Ely Raz @.***> a écrit :

The same error

ex1 <- from_csv_to_Out('E:/Ex1.csv') Error in matrix(., ncol = 2, byrow = T) : 'data' must be of a vector type, was 'NULL'

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MomX/Momocs/issues/211#issuecomment-866824548, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABECQFSFOIZAYGCLBZIZIJLTUHMZPANCNFSM4NDNWFNA .

-- vincentbonhomme.fr http://www.vincentbonhomme.fr

[vbonhomme]

wait a min, where does from_csv_to_Out comes from ??? it's not in the released version of Momocs ! send me your csv I can do the import anyway

Le mer. 23 juin 2021 à 15:16, Vincent Bonhomme @.***> a écrit :

send me "Ex1.csv" > @.***

Le mer. 23 juin 2021 à 15:13, Ely Raz @.***> a écrit :

The same error

ex1 <- from_csv_to_Out('E:/Ex1.csv') Error in matrix(., ncol = 2, byrow = T) : 'data' must be of a vector type, was 'NULL'

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MomX/Momocs/issues/211#issuecomment-866824548, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABECQFSFOIZAYGCLBZIZIJLTUHMZPANCNFSM4NDNWFNA .

-- vincentbonhomme.fr http://www.vincentbonhomme.fr

-- vincentbonhomme.fr http://www.vincentbonhomme.fr

[raz1]

It is the file given above by lrocax

[vbonhomme]

this works like a charm on my machine:

from_csv_to_Out <- function(df, col){

  df <- as_tibble(df) # cosmetics

  fac <- df %>% select(-col)     # all but non


  coo <- df %>% select(col) %>%
    #split rowwise
    split(1:nrow(.)) %>%
    # for each row, turn columns into shape
    map(~.x %>%
      pull() %>%
      str_remove_all("\\[|\\]") %>%
      str_replace_all(";", " ") %>%
      str_split(" ") %>%
      unlist() %>%
      as.numeric() %>%
      matrix(ncol=2, byrow=T)
    )

  # build  and return the Out

  Out(coo, fac)

}

"~/Desktop/Ex1.csv" %>%  read.table(h=T, sep=",") %>% from_csv_to_Out(col=4)

Le mer. 23 juin 2021 à 15:20, Ely Raz @.***> a écrit :

It is the file given above by lrocax

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/MomX/Momocs/issues/211#issuecomment-866829957, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABECQFV7CRISFLB4SIL3DBDTUHNR3ANCNFSM4NDNWFNA .

-- vincentbonhomme.fr http://www.vincentbonhomme.fr

[raz1]

Thanks