A little more information would be useful. Why did it stop? Was there an
error message? Can you show the commands/console log of what you did.
Provide information on how much memory your computer has on it. When the
operation completed, how much memory was used. An important aspect is how
many columns did the data have. How big was the file on disk. What other
objects were in memory at the same time. The list can go on and on, so
more information would be useful to understand the problem.
Data Munger Guru
What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.
On Wed, Jul 12, 2017 at 2:44 AM, Mangalani Peter Makananisa <
[hidden email]> wrote:
> Dear R-Gurus,
> I am trying to read in data with 24,349,113 rows to R-3.3.3 (64 bit) and
> have used the library "data.table" and It managed to read 23,347,070
> rows and the remainder was 2,043 rows only.
> Could you please advise me as to which library/R-commands is suitable to
> read the full data in to R?
> Kind regards,
> Mangalani Peter Makananisa (5786)
> South African Revenue Service (SARS)
> +2782 456 4669 / +2712 422 7357
> Please Note: This email and its contents are subject to our email legal
> notice which can be viewed at http://www.sars.gov.za/Pages/ > Email-disclaimer.aspx
> [[alternative HTML version deleted]]
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/ > posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
In reply to this post by Mangalani Peter Makananisa
> On 12 Jul 2017, at 08:44 , Mangalani Peter Makananisa <[hidden email]> wrote:
> Dear R-Gurus,
> I am trying to read in data with 24,349,113 rows to R-3.3.3 (64 bit) and have used the library "data.table" and It managed to read 23,347,070 rows and the remainder was 2,043 rows only.
I make that 1002043 rows... (typo?). One possibility is that you are skipping lines because of something like quote characters in data or some input lines being one field short. You might also simply be running out of memory.
1) data.table is a contributed package. Its maintainer may be a better point of contact.
2) if you hope to get advice from the R community at large, it would be useful to tell us what you did, what kind of data you try to read using which command, etc. And, as Jim Holtman already said: error messages, if any.
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Office: A 4.23
Email: [hidden email] Priv: [hidden email]