importing data

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

importing data

Ray Cheung
Dear All,

Sorry for asking a newbie question. I want to ask how to import 1000
datasets whose file names are labelled from data1.dat to data1000.dat into
R so that they are named M[1, , ] to M[1000, , ] accordingly. Thank you
very much.

Best Regards,
Ray

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: importing data

Michael Weylandt
On Wed, Jan 23, 2013 at 9:16 AM, Ray Cheung <[hidden email]> wrote:
> Dear All,
>
> Sorry for asking a newbie question. I want to ask how to import 1000
> datasets whose file names are labelled from data1.dat to data1000.dat into
> R so that they are named M[1, , ] to M[1000, , ] accordingly. Thank you
> very much.
>,

Hi Ray,

I'm pretty sure you don't mean "named" M[1,,] etc. but rather that
there's only one object M and that's how the slices come into
existence:

What you'll want to do is something like this:

little_helpful_function(n){
    file_name <- paste("data", n, ".dat", sep = "")
    read.dta(file_name, ##OTHER PARAMETERS)
}

list_of_datasets <- lapply(1:1000, little_helpful_function)

output <- do.call(c, list_of_datasets)

dim(output) <- c(dim(list_of_datasets[[1]]), 1000)

Or something like that. Note that I'm not quite sure what a dta file
is, so I'll leave it to you to find an appropriate read.dta function.

Feel free to write back (cc'ing the list) if you don't understand all
of the above.

Cheers,
Michael

> Best Regards,
> Ray
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: importing data

Ray Cheung
Dear Michael,

Thanks for your codes. However, lapply does not work in my case since I've
some files missing in the data (say, the file data101.dat). Do you have any
suggestions on this?? Thank you very much.

Best Wishes,
Ray

On Wed, Jan 23, 2013 at 8:15 PM, R. Michael Weylandt <
[hidden email]> wrote:

> On Wed, Jan 23, 2013 at 9:16 AM, Ray Cheung <[hidden email]> wrote:
> > Dear All,
> >
> > Sorry for asking a newbie question. I want to ask how to import 1000
> > datasets whose file names are labelled from data1.dat to data1000.dat
> into
> > R so that they are named M[1, , ] to M[1000, , ] accordingly. Thank you
> > very much.
> >,
>
> Hi Ray,
>
> I'm pretty sure you don't mean "named" M[1,,] etc. but rather that
> there's only one object M and that's how the slices come into
> existence:
>
> What you'll want to do is something like this:
>
> little_helpful_function(n){
>     file_name <- paste("data", n, ".dat", sep = "")
>     read.dta(file_name, ##OTHER PARAMETERS)
> }
>
> list_of_datasets <- lapply(1:1000, little_helpful_function)
>
> output <- do.call(c, list_of_datasets)
>
> dim(output) <- c(dim(list_of_datasets[[1]]), 1000)
>
> Or something like that. Note that I'm not quite sure what a dta file
> is, so I'll leave it to you to find an appropriate read.dta function.
>
> Feel free to write back (cc'ing the list) if you don't understand all
> of the above.
>
> Cheers,
> Michael
>
> > Best Regards,
> > Ray
> >
> >         [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: importing data

Michael Weylandt
On Fri, Jan 25, 2013 at 6:11 AM, Ray Cheung <[hidden email]> wrote:
> Dear Michael,
>
> Thanks for your codes. However, lapply does not work in my case since I've
> some files missing in the data (say, the file data101.dat). Do you have any
> suggestions on this?? Thank you very much.
>

You could simply add a test using file.exists() but I'm not sure what
you want to do with the M matrix then -- omit the slice (so the others
are all shifted down one) or fill it entirely with NA's.

Michael

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: importing data

Ivan Calandra-2
Hi,

Not sure this is what you need, but what about list.files()?
It can get you all the files from a given folder, and you could then
work this list with regular expressions for example.

HTH,
Ivan

--
Ivan CALANDRA
Université de Bourgogne
UMR CNRS/uB 6282 Biogéosciences
6 Boulevard Gabriel
21000 Dijon, FRANCE
+33(0)3.80.39.63.06
[hidden email]
http://biogeosciences.u-bourgogne.fr/calandra

Le 25/01/13 10:00, R. Michael Weylandt a écrit :

> On Fri, Jan 25, 2013 at 6:11 AM, Ray Cheung <[hidden email]> wrote:
>> Dear Michael,
>>
>> Thanks for your codes. However, lapply does not work in my case since I've
>> some files missing in the data (say, the file data101.dat). Do you have any
>> suggestions on this?? Thank you very much.
>>
> You could simply add a test using file.exists() but I'm not sure what
> you want to do with the M matrix then -- omit the slice (so the others
> are all shifted down one) or fill it entirely with NA's.
>
> Michael
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: importing data

Ray Cheung
Thanks for your commands, Ivan and Michael! However, I am still not
producing the right codes. Would you please help me on this? I've written
the following codes. Please comment. Thank you very much.

Task: Reading data1.dat to data1000.dat (with missing files) into R.
Missing files can be omitted in the list.

###FUNCTION TO READ FILES
little_helpful <- function(n) {
file_name <- paste0("C:/.../data", n, ".dat")
read.table(file_name)
}

###RETURN AN OBJECT WHICH CHECKS FOR THE EXISTENCE OF FILES
check  <- function(n) {
a <- ifelse(file.exists(paste0("C:/.../data", n, ".dat")), 1, 0)
a
}
###Combining the functions
IMPORT <- function(n) {
   L <- check(1:n)
   for (i in 1:n) {
      if (L[i] == 1)
      list_of_datasets <- lapply(i, little_helpful) else list_of_datasets
<- 0
      }
   list_of_datasets
   }

Thanks for all comments.

Best Regards,
Ray

On Fri, Jan 25, 2013 at 5:48 PM, Ivan Calandra <[hidden email]
> wrote:

> Hi,
>
> Not sure this is what you need, but what about list.files()?
> It can get you all the files from a given folder, and you could then work
> this list with regular expressions for example.
>
> HTH,
> Ivan
>
> --
> Ivan CALANDRA
> Université de Bourgogne
> UMR CNRS/uB 6282 Biogéosciences
> 6 Boulevard Gabriel
> 21000 Dijon, FRANCE
> +33(0)3.80.39.63.06
> [hidden email]
> http://biogeosciences.u-**bourgogne.fr/calandra<http://biogeosciences.u-bourgogne.fr/calandra>
>
> Le 25/01/13 10:00, R. Michael Weylandt a écrit :
>
>> On Fri, Jan 25, 2013 at 6:11 AM, Ray Cheung <[hidden email]> wrote:
>>
>>> Dear Michael,
>>>
>>> Thanks for your codes. However, lapply does not work in my case since
>>> I've
>>> some files missing in the data (say, the file data101.dat). Do you have
>>> any
>>> suggestions on this?? Thank you very much.
>>>
>>>  You could simply add a test using file.exists() but I'm not sure what
>> you want to do with the M matrix then -- omit the slice (so the others
>> are all shifted down one) or fill it entirely with NA's.
>>
>> Michael
>>
>> ______________________________**________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/**listinfo/r-help<https://stat.ethz.ch/mailman/listinfo/r-help>
>> PLEASE do read the posting guide http://www.R-project.org/**
>> posting-guide.html <http://www.R-project.org/posting-guide.html>
>> and provide commented, minimal, self-contained, reproducible code.
>>
>>
> ______________________________**________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/**listinfo/r-help<https://stat.ethz.ch/mailman/listinfo/r-help>
> PLEASE do read the posting guide http://www.R-project.org/**
> posting-guide.html <http://www.R-project.org/posting-guide.html>
> and provide commented, minimal, self-contained, reproducible code.
>
        [[alternative HTML version deleted]]


______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: importing data

David Hugh-Jones-3
Hi Ray,
Comments below:

On 26 January 2013 09:03, Ray Cheung <[hidden email]> wrote:
[snip]

> ###FUNCTION TO READ FILES
> little_helpful <- function(n) {
> file_name <- paste0("C:/.../data", n, ".dat")
> read.table(file_name)
> }
>
> ###RETURN AN OBJECT WHICH CHECKS FOR THE EXISTENCE OF FILES
> check  <- function(n) {
> a <- ifelse(file.exists(paste0("C:/.../data", n, ".dat")), 1, 0)
> a
> }

Too complex. Why not just use file.exists directly?

> ###Combining the functions
> IMPORT <- function(n) {
>    L <- check(1:n)
>    for (i in 1:n) {
>       if (L[i] == 1)
>       list_of_datasets <- lapply(i, little_helpful) else list_of_datasets
> <- 0
>       }
>    list_of_datasets
>    }
>

Too complex here too. I suggest something like:

M <- list()
for (i in 1:n) {
> file_name <- paste0("C:/.../data", n, ".dat")
  if (file.exists(file_name)) M[i] <- read.table(file_name)
}

R gurus don't like for() loops, but they are easy for humans to understand.
If this doesn't work, post the error message.


> Thanks for all comments.
>
> Best Regards,
> Ray
>
> On Fri, Jan 25, 2013 at 5:48 PM, Ivan Calandra <[hidden email]
>> wrote:
>
>> Hi,
>>
>> Not sure this is what you need, but what about list.files()?
>> It can get you all the files from a given folder, and you could then work
>> this list with regular expressions for example.
>>
>> HTH,
>> Ivan
>>
>> --
>> Ivan CALANDRA
>> Université de Bourgogne
>> UMR CNRS/uB 6282 Biogéosciences
>> 6 Boulevard Gabriel
>> 21000 Dijon, FRANCE
>> +33(0)3.80.39.63.06
>> [hidden email]
>> http://biogeosciences.u-**bourgogne.fr/calandra<http://biogeosciences.u-bourgogne.fr/calandra>
>>
>> Le 25/01/13 10:00, R. Michael Weylandt a écrit :
>>
>>> On Fri, Jan 25, 2013 at 6:11 AM, Ray Cheung <[hidden email]> wrote:
>>>
>>>> Dear Michael,
>>>>
>>>> Thanks for your codes. However, lapply does not work in my case since
>>>> I've
>>>> some files missing in the data (say, the file data101.dat). Do you have
>>>> any
>>>> suggestions on this?? Thank you very much.
>>>>
>>>>  You could simply add a test using file.exists() but I'm not sure what
>>> you want to do with the M matrix then -- omit the slice (so the others
>>> are all shifted down one) or fill it entirely with NA's.
>>>
>>> Michael
>>>
>>> ______________________________**________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/**listinfo/r-help<https://stat.ethz.ch/mailman/listinfo/r-help>
>>> PLEASE do read the posting guide http://www.R-project.org/**
>>> posting-guide.html <http://www.R-project.org/posting-guide.html>
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>>
>> ______________________________**________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/**listinfo/r-help<https://stat.ethz.ch/mailman/listinfo/r-help>
>> PLEASE do read the posting guide http://www.R-project.org/**
>> posting-guide.html <http://www.R-project.org/posting-guide.html>
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
>         [[alternative HTML version deleted]]
>
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: importing data

Ivan Calandra-2
In reply to this post by Ray Cheung
Hi Ray!

I'm insisting with list.files...!

What about like this (untested)?
file_names <- list.files(path="C:/.../data", pattern=".dat$",
full.names=TRUE)
list_of_dataset <- do.call(read.table, file_names)

Let me know if this helps!
Ivan

--
Ivan CALANDRA
Université de Bourgogne
UMR CNRS/uB 6282 Biogéosciences
6 Boulevard Gabriel
21000 Dijon, FRANCE
+33(0)3.80.39.63.06
[hidden email]
http://biogeosciences.u-bourgogne.fr/calandra

Le 26/01/13 10:03, Ray Cheung a écrit :

> Thanks for your commands, Ivan and Michael! However, I am still not
> producing the right codes. Would you please help me on this? I've
> written the following codes. Please comment. Thank you very much.
> Task: Reading data1.dat to data1000.dat (with missing files) into R.
> Missing files can be omitted in the list.
> ###FUNCTION TO READ FILES
> little_helpful <- function(n) {
> file_name <- paste0("C:/.../data", n, ".dat")
> read.table(file_name)
> }
> ###RETURN AN OBJECT WHICH CHECKS FOR THE EXISTENCE OF FILES
> check  <- function(n) {
> a <- ifelse(file.exists(paste0("C:/.../data", n, ".dat")), 1, 0)
> a
> }
> ###Combining the functions
> IMPORT <- function(n) {
>    L <- check(1:n)
>    for (i in 1:n) {
>       if (L[i] == 1)
>       list_of_datasets <- lapply(i, little_helpful) else
> list_of_datasets <- 0
>       }
>    list_of_datasets
>    }
> Thanks for all comments.
> Best Regards,
> Ray
>
> On Fri, Jan 25, 2013 at 5:48 PM, Ivan Calandra
> <[hidden email] <mailto:[hidden email]>>
> wrote:
>
>     Hi,
>
>     Not sure this is what you need, but what about list.files()?
>     It can get you all the files from a given folder, and you could
>     then work this list with regular expressions for example.
>
>     HTH,
>     Ivan
>
>     --
>     Ivan CALANDRA
>     Université de Bourgogne
>     UMR CNRS/uB 6282 Biogéosciences
>     6 Boulevard Gabriel
>     21000 Dijon, FRANCE
>     +33(0)3.80.39.63.06 <tel:%2B33%280%293.80.39.63.06>
>     [hidden email] <mailto:[hidden email]>
>     http://biogeosciences.u-bourgogne.fr/calandra
>
>     Le 25/01/13 10:00, R. Michael Weylandt a écrit :
>
>         On Fri, Jan 25, 2013 at 6:11 AM, Ray Cheung <[hidden email]
>         <mailto:[hidden email]>> wrote:
>
>             Dear Michael,
>
>             Thanks for your codes. However, lapply does not work in my
>             case since I've
>             some files missing in the data (say, the file
>             data101.dat). Do you have any
>             suggestions on this?? Thank you very much.
>
>         You could simply add a test using file.exists() but I'm not
>         sure what
>         you want to do with the M matrix then -- omit the slice (so
>         the others
>         are all shifted down one) or fill it entirely with NA's.
>
>         Michael
>
>         ______________________________________________
>         [hidden email] <mailto:[hidden email]> mailing list
>         https://stat.ethz.ch/mailman/listinfo/r-help
>         PLEASE do read the posting guide
>         http://www.R-project.org/posting-guide.html
>         and provide commented, minimal, self-contained, reproducible code.
>
>
>     ______________________________________________
>     [hidden email] <mailto:[hidden email]> mailing list
>     https://stat.ethz.ch/mailman/listinfo/r-help
>     PLEASE do read the posting guide
>     http://www.R-project.org/posting-guide.html
>     and provide commented, minimal, self-contained, reproducible code.
>
>
        [[alternative HTML version deleted]]


______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: importing data

Ray Cheung
Thanks a million for all help provided!! I can do what I intend to using
the "for loop". However, I'm still eager to try the list.files approach.
Here is the error message that I got using Ivan's code:

> list_of_dataset <- do.call(read.table, file_names)
Error in do.call(read.table, file_names) : second argument must be a list

Please advise.

Ray

On Sun, Jan 27, 2013 at 10:57 PM, Ivan Calandra <
[hidden email]> wrote:

>  Hi Ray!
>
> I'm insisting with list.files...!
>
> What about like this (untested)?
> file_names <- list.files(path="C:/.../data", pattern=".dat$",
> full.names=TRUE)
> list_of_dataset <- do.call(read.table, file_names)
>
> Let me know if this helps!
> Ivan
>
> --
> Ivan CALANDRA
> Université de Bourgogne
> UMR CNRS/uB 6282 Biogéosciences
> 6 Boulevard Gabriel
> 21000 Dijon, FRANCE+33(0)[hidden email]
> http://biogeosciences.u-bourgogne.fr/calandra
>
> Le 26/01/13 10:03, Ray Cheung a écrit :
>
> Thanks for your commands, Ivan and Michael! However, I am still not
> producing the right codes. Would you please help me on this? I've written
> the following codes. Please comment. Thank you very much.
>
> Task: Reading data1.dat to data1000.dat (with missing files) into R.
> Missing files can be omitted in the list.
>
> ###FUNCTION TO READ FILES
> little_helpful <- function(n) {
> file_name <- paste0("C:/.../data", n, ".dat")
> read.table(file_name)
> }
>
> ###RETURN AN OBJECT WHICH CHECKS FOR THE EXISTENCE OF FILES
> check  <- function(n) {
> a <- ifelse(file.exists(paste0("C:/.../data", n, ".dat")), 1, 0)
> a
> }
>  ###Combining the functions
> IMPORT <- function(n) {
>    L <- check(1:n)
>    for (i in 1:n) {
>       if (L[i] == 1)
>       list_of_datasets <- lapply(i, little_helpful) else list_of_datasets
> <- 0
>       }
>    list_of_datasets
>    }
>
> Thanks for all comments.
>
> Best Regards,
> Ray
>
>  On Fri, Jan 25, 2013 at 5:48 PM, Ivan Calandra <
> [hidden email]> wrote:
>
>> Hi,
>>
>> Not sure this is what you need, but what about list.files()?
>> It can get you all the files from a given folder, and you could then work
>> this list with regular expressions for example.
>>
>> HTH,
>> Ivan
>>
>> --
>> Ivan CALANDRA
>> Université de Bourgogne
>> UMR CNRS/uB 6282 Biogéosciences
>> 6 Boulevard Gabriel
>> 21000 Dijon, FRANCE
>> +33(0)3.80.39.63.06
>> [hidden email]
>> http://biogeosciences.u-bourgogne.fr/calandra
>>
>> Le 25/01/13 10:00, R. Michael Weylandt a écrit :
>>
>>>  On Fri, Jan 25, 2013 at 6:11 AM, Ray Cheung <[hidden email]> wrote:
>>>
>>>> Dear Michael,
>>>>
>>>> Thanks for your codes. However, lapply does not work in my case since
>>>> I've
>>>> some files missing in the data (say, the file data101.dat). Do you have
>>>> any
>>>> suggestions on this?? Thank you very much.
>>>>
>>>>  You could simply add a test using file.exists() but I'm not sure what
>>> you want to do with the M matrix then -- omit the slice (so the others
>>> are all shifted down one) or fill it entirely with NA's.
>>>
>>> Michael
>>>
>>>   ______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
>
        [[alternative HTML version deleted]]


______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: importing data

Ivan Calandra-2
What about this then:
list_of_datasets <- lapply(file_names, read.table, other_args_to_read.table)

Something that might then be useful is:
names(list_of_datasets) <- file_names

Does it do it now?

Ivan

--
Ivan CALANDRA
Université de Bourgogne
UMR CNRS/uB 6282 Biogéosciences
6 Boulevard Gabriel
21000 Dijon, FRANCE
+33(0)3.80.39.63.06
[hidden email]
http://biogeosciences.u-bourgogne.fr/calandra

Le 28/01/13 07:34, Ray Cheung a écrit :

> Thanks a million for all help provided!! I can do what I intend to
> using the "for loop". However, I'm still eager to try the list.files
> approach. Here is the error message that I got using Ivan's code:
>
> > list_of_dataset <- do.call(read.table, file_names)
> Error in do.call(read.table, file_names) : second argument must be a list
>
> Please advise.
>
> Ray
>
> On Sun, Jan 27, 2013 at 10:57 PM, Ivan Calandra
> <[hidden email] <mailto:[hidden email]>>
> wrote:
>
>     Hi Ray!
>
>     I'm insisting with list.files...!
>
>     What about like this (untested)?
>     file_names <- list.files(path="C:/.../data", pattern=".dat$",
>     full.names=TRUE)
>     list_of_dataset <- do.call(read.table, file_names)
>
>     Let me know if this helps!
>     Ivan
>
>     -- Ivan CALANDRA Université de Bourgogne UMR CNRS/uB 6282
>     Biogéosciences 6 Boulevard Gabriel 21000 Dijon, FRANCE
>     +33(0)3.80.39.63.06 <tel:%2B33%280%293.80.39.63.06>
>     [hidden email] <mailto:[hidden email]>
>     http://biogeosciences.u-bourgogne.fr/calandra
>
>     Le 26/01/13 10:03, Ray Cheung a écrit :
>>     Thanks for your commands, Ivan and Michael! However, I am still
>>     not producing the right codes. Would you please help me on
>>     this? I've written the following codes. Please comment. Thank you
>>     very much.
>>     Task: Reading data1.dat to data1000.dat (with missing files) into
>>     R. Missing files can be omitted in the list.
>>     ###FUNCTION TO READ FILES
>>     little_helpful <- function(n) {
>>     file_name <- paste0("C:/.../data", n, ".dat")
>>     read.table(file_name)
>>     }
>>     ###RETURN AN OBJECT WHICH CHECKS FOR THE EXISTENCE OF FILES
>>     check  <- function(n) {
>>     a <- ifelse(file.exists(paste0("C:/.../data", n, ".dat")), 1, 0)
>>     a
>>     }
>>     ###Combining the functions
>>     IMPORT <- function(n) {
>>        L <- check(1:n)
>>        for (i in 1:n) {
>>           if (L[i] == 1)
>>           list_of_datasets <- lapply(i, little_helpful) else
>>     list_of_datasets <- 0
>>           }
>>        list_of_datasets
>>        }
>>     Thanks for all comments.
>>     Best Regards,
>>     Ray
>>
>>     On Fri, Jan 25, 2013 at 5:48 PM, Ivan Calandra
>>     <[hidden email]
>>     <mailto:[hidden email]>> wrote:
>>
>>         Hi,
>>
>>         Not sure this is what you need, but what about list.files()?
>>         It can get you all the files from a given folder, and you
>>         could then work this list with regular expressions for example.
>>
>>         HTH,
>>         Ivan
>>
>>         --
>>         Ivan CALANDRA
>>         Université de Bourgogne
>>         UMR CNRS/uB 6282 Biogéosciences
>>         6 Boulevard Gabriel
>>         21000 Dijon, FRANCE
>>         +33(0)3.80.39.63.06 <tel:%2B33%280%293.80.39.63.06>
>>         [hidden email]
>>         <mailto:[hidden email]>
>>         http://biogeosciences.u-bourgogne.fr/calandra
>>
>>         Le 25/01/13 10:00, R. Michael Weylandt a écrit :
>>
>>             On Fri, Jan 25, 2013 at 6:11 AM, Ray Cheung
>>             <[hidden email] <mailto:[hidden email]>> wrote:
>>
>>                 Dear Michael,
>>
>>                 Thanks for your codes. However, lapply does not work
>>                 in my case since I've
>>                 some files missing in the data (say, the file
>>                 data101.dat). Do you have any
>>                 suggestions on this?? Thank you very much.
>>
>>             You could simply add a test using file.exists() but I'm
>>             not sure what
>>             you want to do with the M matrix then -- omit the slice
>>             (so the others
>>             are all shifted down one) or fill it entirely with NA's.
>>
>>             Michael
>>
>>             ______________________________________________
>>             [hidden email] <mailto:[hidden email]>
>>             mailing list
>>             https://stat.ethz.ch/mailman/listinfo/r-help
>>             PLEASE do read the posting guide
>>             http://www.R-project.org/posting-guide.html
>>             and provide commented, minimal, self-contained,
>>             reproducible code.
>>
>>
>>         ______________________________________________
>>         [hidden email] <mailto:[hidden email]> mailing list
>>         https://stat.ethz.ch/mailman/listinfo/r-help
>>         PLEASE do read the posting guide
>>         http://www.R-project.org/posting-guide.html
>>         and provide commented, minimal, self-contained, reproducible
>>         code.
>>
>>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: importing data

Ray Cheung
Dear Ivan,

It works perfectly fine now. I love this code more since I need not delete
the NULL list myself (and it should be faster, right?). Thank you very much
for your help!

cheers,
Ray

On Mon, Jan 28, 2013 at 5:32 PM, Ivan Calandra <[hidden email]
> wrote:

> What about this then:
> list_of_datasets <- lapply(file_names, read.table,
> other_args_to_read.table)
>
> Something that might then be useful is:
> names(list_of_datasets) <- file_names
>
> Does it do it now?
>
>
> Ivan
>
> --
> Ivan CALANDRA
> Université de Bourgogne
> UMR CNRS/uB 6282 Biogéosciences
> 6 Boulevard Gabriel
> 21000 Dijon, FRANCE
> +33(0)3.80.39.63.06
> [hidden email]
> http://biogeosciences.u-**bourgogne.fr/calandra<http://biogeosciences.u-bourgogne.fr/calandra>
>
> Le 28/01/13 07:34, Ray Cheung a écrit :
>
>> Thanks a million for all help provided!! I can do what I intend to using
>> the "for loop". However, I'm still eager to try the list.files approach.
>> Here is the error message that I got using Ivan's code:
>>
>> > list_of_dataset <- do.call(read.table, file_names)
>> Error in do.call(read.table, file_names) : second argument must be a list
>>
>> Please advise.
>>
>> Ray
>>
>> On Sun, Jan 27, 2013 at 10:57 PM, Ivan Calandra <
>> [hidden email] <mailto:ivan.calandra@u-**bourgogne.fr<[hidden email]>>>
>> wrote:
>>
>>     Hi Ray!
>>
>>     I'm insisting with list.files...!
>>
>>     What about like this (untested)?
>>     file_names <- list.files(path="C:/.../data", pattern=".dat$",
>>     full.names=TRUE)
>>     list_of_dataset <- do.call(read.table, file_names)
>>
>>     Let me know if this helps!
>>     Ivan
>>
>>     -- Ivan CALANDRA Université de Bourgogne UMR CNRS/uB 6282
>>     Biogéosciences 6 Boulevard Gabriel 21000 Dijon, FRANCE
>>     +33(0)3.80.39.63.06 <tel:%2B33%280%293.80.39.63.**06>
>>     [hidden email] <mailto:ivan.calandra@u-**bourgogne.fr<[hidden email]>
>> >
>>
>>     http://biogeosciences.u-**bourgogne.fr/calandra<http://biogeosciences.u-bourgogne.fr/calandra>
>>
>>     Le 26/01/13 10:03, Ray Cheung a écrit :
>>
>>>     Thanks for your commands, Ivan and Michael! However, I am still
>>>     not producing the right codes. Would you please help me on
>>>     this? I've written the following codes. Please comment. Thank you
>>>     very much.
>>>     Task: Reading data1.dat to data1000.dat (with missing files) into
>>>     R. Missing files can be omitted in the list.
>>>     ###FUNCTION TO READ FILES
>>>     little_helpful <- function(n) {
>>>     file_name <- paste0("C:/.../data", n, ".dat")
>>>     read.table(file_name)
>>>     }
>>>     ###RETURN AN OBJECT WHICH CHECKS FOR THE EXISTENCE OF FILES
>>>     check  <- function(n) {
>>>     a <- ifelse(file.exists(paste0("C:/**.../data", n, ".dat")), 1, 0)
>>>     a
>>>     }
>>>     ###Combining the functions
>>>     IMPORT <- function(n) {
>>>        L <- check(1:n)
>>>        for (i in 1:n) {
>>>           if (L[i] == 1)
>>>           list_of_datasets <- lapply(i, little_helpful) else
>>>     list_of_datasets <- 0
>>>           }
>>>        list_of_datasets
>>>        }
>>>     Thanks for all comments.
>>>     Best Regards,
>>>     Ray
>>>
>>>     On Fri, Jan 25, 2013 at 5:48 PM, Ivan Calandra
>>>     <[hidden email]
>>>     <mailto:ivan.calandra@u-**bourgogne.fr<[hidden email]>>>
>>> wrote:
>>>
>>>         Hi,
>>>
>>>         Not sure this is what you need, but what about list.files()?
>>>         It can get you all the files from a given folder, and you
>>>         could then work this list with regular expressions for example.
>>>
>>>         HTH,
>>>         Ivan
>>>
>>>         --
>>>         Ivan CALANDRA
>>>         Université de Bourgogne
>>>         UMR CNRS/uB 6282 Biogéosciences
>>>         6 Boulevard Gabriel
>>>         21000 Dijon, FRANCE
>>>         +33(0)3.80.39.63.06 <tel:%2B33%280%293.80.39.63.**06>
>>>         [hidden email]
>>>         <mailto:ivan.calandra@u-**bourgogne.fr<[hidden email]>
>>> >
>>>
>>>         http://biogeosciences.u-**bourgogne.fr/calandra<http://biogeosciences.u-bourgogne.fr/calandra>
>>>
>>>         Le 25/01/13 10:00, R. Michael Weylandt a écrit :
>>>
>>>             On Fri, Jan 25, 2013 at 6:11 AM, Ray Cheung
>>>             <[hidden email] <mailto:[hidden email]>> wrote:
>>>
>>>                 Dear Michael,
>>>
>>>                 Thanks for your codes. However, lapply does not work
>>>                 in my case since I've
>>>                 some files missing in the data (say, the file
>>>                 data101.dat). Do you have any
>>>                 suggestions on this?? Thank you very much.
>>>
>>>             You could simply add a test using file.exists() but I'm
>>>             not sure what
>>>             you want to do with the M matrix then -- omit the slice
>>>             (so the others
>>>             are all shifted down one) or fill it entirely with NA's.
>>>
>>>             Michael
>>>
>>>             ______________________________**________________
>>>             [hidden email] <mailto:[hidden email]>
>>>
>>>             mailing list
>>>             https://stat.ethz.ch/mailman/**listinfo/r-help<https://stat.ethz.ch/mailman/listinfo/r-help>
>>>             PLEASE do read the posting guide
>>>             http://www.R-project.org/**posting-guide.html<http://www.R-project.org/posting-guide.html>
>>>             and provide commented, minimal, self-contained,
>>>             reproducible code.
>>>
>>>
>>>         ______________________________**________________
>>>         [hidden email] <mailto:[hidden email]> mailing list
>>>
>>>         https://stat.ethz.ch/mailman/**listinfo/r-help<https://stat.ethz.ch/mailman/listinfo/r-help>
>>>         PLEASE do read the posting guide
>>>         http://www.R-project.org/**posting-guide.html<http://www.R-project.org/posting-guide.html>
>>>         and provide commented, minimal, self-contained, reproducible
>>>         code.
>>>
>>>
>>>
>>
>
        [[alternative HTML version deleted]]


______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: importing data

Ivan Calandra-2
This code is indeed much shorter. About the speed, I guess it should be
faster, but you should test it with system.time()

I'm glad that it helped.
Ivan


--
Ivan CALANDRA
Université de Bourgogne
UMR CNRS/uB 6282 Biogéosciences
6 Boulevard Gabriel
21000 Dijon, FRANCE
+33(0)3.80.39.63.06
[hidden email]
http://biogeosciences.u-bourgogne.fr/calandra

Le 28/01/13 16:38, Ray Cheung a écrit :

> Dear Ivan,
> It works perfectly fine now. I love this code more since I need not
> delete the NULL list myself (and it should be faster, right?). Thank
> you very much for your help!
> cheers,
> Ray
>
> On Mon, Jan 28, 2013 at 5:32 PM, Ivan Calandra
> <[hidden email] <mailto:[hidden email]>>
> wrote:
>
>     What about this then:
>     list_of_datasets <- lapply(file_names, read.table,
>     other_args_to_read.table)
>
>     Something that might then be useful is:
>     names(list_of_datasets) <- file_names
>
>     Does it do it now?
>
>
>     Ivan
>
>     --
>     Ivan CALANDRA
>     Université de Bourgogne
>     UMR CNRS/uB 6282 Biogéosciences
>     6 Boulevard Gabriel
>     21000 Dijon, FRANCE
>     +33(0)3.80.39.63.06 <tel:%2B33%280%293.80.39.63.06>
>     [hidden email] <mailto:[hidden email]>
>     http://biogeosciences.u-bourgogne.fr/calandra
>
>     Le 28/01/13 07:34, Ray Cheung a écrit :
>
>         Thanks a million for all help provided!! I can do what I
>         intend to using the "for loop". However, I'm still eager to
>         try the list.files approach. Here is the error message that I
>         got using Ivan's code:
>
>         > list_of_dataset <- do.call(read.table, file_names)
>         Error in do.call(read.table, file_names) : second argument
>         must be a list
>
>         Please advise.
>
>         Ray
>
>         On Sun, Jan 27, 2013 at 10:57 PM, Ivan Calandra
>         <[hidden email]
>         <mailto:[hidden email]>
>         <mailto:[hidden email]
>         <mailto:[hidden email]>>> wrote:
>
>             Hi Ray!
>
>             I'm insisting with list.files...!
>
>             What about like this (untested)?
>             file_names <- list.files(path="C:/.../data", pattern=".dat$",
>             full.names=TRUE)
>             list_of_dataset <- do.call(read.table, file_names)
>
>             Let me know if this helps!
>             Ivan
>
>             -- Ivan CALANDRA Université de Bourgogne UMR CNRS/uB 6282
>             Biogéosciences 6 Boulevard Gabriel 21000 Dijon, FRANCE
>         +33(0)3.80.39.63.06 <tel:%2B33%280%293.80.39.63.06>
>         <tel:%2B33%280%293.80.39.63.06>
>         [hidden email]
>         <mailto:[hidden email]>
>         <mailto:[hidden email]
>         <mailto:[hidden email]>>
>
>         http://biogeosciences.u-bourgogne.fr/calandra
>
>             Le 26/01/13 10:03, Ray Cheung a écrit :
>
>                 Thanks for your commands, Ivan and Michael! However, I
>             am still
>                 not producing the right codes. Would you please help me on
>                 this? I've written the following codes. Please
>             comment. Thank you
>                 very much.
>                 Task: Reading data1.dat to data1000.dat (with missing
>             files) into
>                 R. Missing files can be omitted in the list.
>                 ###FUNCTION TO READ FILES
>                 little_helpful <- function(n) {
>                 file_name <- paste0("C:/.../data", n, ".dat")
>                 read.table(file_name)
>                 }
>                 ###RETURN AN OBJECT WHICH CHECKS FOR THE EXISTENCE OF
>             FILES
>                 check  <- function(n) {
>                 a <- ifelse(file.exists(paste0("C:/.../data", n,
>             ".dat")), 1, 0)
>                 a
>                 }
>                 ###Combining the functions
>                 IMPORT <- function(n) {
>                    L <- check(1:n)
>                    for (i in 1:n) {
>                       if (L[i] == 1)
>                       list_of_datasets <- lapply(i, little_helpful) else
>                 list_of_datasets <- 0
>                       }
>                    list_of_datasets
>                    }
>                 Thanks for all comments.
>                 Best Regards,
>                 Ray
>
>                 On Fri, Jan 25, 2013 at 5:48 PM, Ivan Calandra
>                 <[hidden email]
>             <mailto:[hidden email]>
>                 <mailto:[hidden email]
>             <mailto:[hidden email]>>> wrote:
>
>                     Hi,
>
>                     Not sure this is what you need, but what about
>             list.files()?
>                     It can get you all the files from a given folder,
>             and you
>                     could then work this list with regular expressions
>             for example.
>
>                     HTH,
>                     Ivan
>
>                     --
>                     Ivan CALANDRA
>                     Université de Bourgogne
>                     UMR CNRS/uB 6282 Biogéosciences
>                     6 Boulevard Gabriel
>                     21000 Dijon, FRANCE
>             +33(0)3.80.39.63.06 <tel:%2B33%280%293.80.39.63.06>
>             <tel:%2B33%280%293.80.39.63.06>
>             [hidden email]
>             <mailto:[hidden email]>
>                     <mailto:[hidden email]
>             <mailto:[hidden email]>>
>
>             http://biogeosciences.u-bourgogne.fr/calandra
>
>                     Le 25/01/13 10:00, R. Michael Weylandt a écrit :
>
>                         On Fri, Jan 25, 2013 at 6:11 AM, Ray Cheung
>                         <[hidden email] <mailto:[hidden email]>
>             <mailto:[hidden email] <mailto:[hidden email]>>> wrote:
>
>                             Dear Michael,
>
>                             Thanks for your codes. However, lapply
>             does not work
>                             in my case since I've
>                             some files missing in the data (say, the file
>                             data101.dat). Do you have any
>                             suggestions on this?? Thank you very much.
>
>                         You could simply add a test using
>             file.exists() but I'm
>                         not sure what
>                         you want to do with the M matrix then -- omit
>             the slice
>                         (so the others
>                         are all shifted down one) or fill it entirely
>             with NA's.
>
>                         Michael
>
>                         ______________________________________________
>             [hidden email] <mailto:[hidden email]>
>             <mailto:[hidden email] <mailto:[hidden email]>>
>
>                         mailing list
>             https://stat.ethz.ch/mailman/listinfo/r-help
>                         PLEASE do read the posting guide
>             http://www.R-project.org/posting-guide.html
>                         and provide commented, minimal, self-contained,
>                         reproducible code.
>
>
>                     ______________________________________________
>             [hidden email] <mailto:[hidden email]>
>             <mailto:[hidden email]
>             <mailto:[hidden email]>> mailing list
>
>             https://stat.ethz.ch/mailman/listinfo/r-help
>                     PLEASE do read the posting guide
>             http://www.R-project.org/posting-guide.html
>                     and provide commented, minimal, self-contained,
>             reproducible
>                     code.
>
>
>
>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.