taking a small piece of large tiff

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

taking a small piece of large tiff

R help mailing list-2
Dear Forum 
I am trying to cut out a small section of a very large 2-dimensional grayscale image as a tiff in R, but it is having difficulty handling such large files.  I have looked at bigmemory and ff packages but it is unclear how I can use these packages with tiffs. Can anyone please suggest something? I have tried tiff and rtiff libraries.
Thanks in advance.
        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: taking a small piece of large tiff

jholtman
How big is 'large'?

Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.


On Tue, Apr 4, 2017 at 7:47 AM, Louisa Reynolds via R-help
<[hidden email]> wrote:

> Dear Forum
> I am trying to cut out a small section of a very large 2-dimensional grayscale image as a tiff in R, but it is having difficulty handling such large files.  I have looked at bigmemory and ff packages but it is unclear how I can use these packages with tiffs. Can anyone please suggest something? I have tried tiff and rtiff libraries.
> Thanks in advance.
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: taking a small piece of large tiff

R help mailing list-2
Ok. I have a tiff of size over 2GB. It covers a sixth of the Earth's surface and I'm trying to cut a UK piece out of it. The tiff I start with seems to be too large for R to handle.


Sent from my iPhone

> On 4 Apr 2017, at 18:37, jim holtman <[hidden email]> wrote:
>
> How big is 'large'?
>
> Jim Holtman
> Data Munger Guru
>
> What is the problem that you are trying to solve?
> Tell me what you want to do, not how you want to do it.
>
>
> On Tue, Apr 4, 2017 at 7:47 AM, Louisa Reynolds via R-help
> <[hidden email]> wrote:
>> Dear Forum
>> I am trying to cut out a small section of a very large 2-dimensional grayscale image as a tiff in R, but it is having difficulty handling such large files.  I have looked at bigmemory and ff packages but it is unclear how I can use these packages with tiffs. Can anyone please suggest something? I have tried tiff and rtiff libraries.
>> Thanks in advance.
>>        [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: taking a small piece of large tiff

jholtman
if you have 8GB of memory it should be easy to handle.


Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?
Tell me what you want to do, not how you want to do it.


On Wed, Apr 5, 2017 at 3:23 AM, Louisa Reynolds
<[hidden email]> wrote:

> Ok. I have a tiff of size over 2GB. It covers a sixth of the Earth's surface and I'm trying to cut a UK piece out of it. The tiff I start with seems to be too large for R to handle.
>
>
> Sent from my iPhone
>
>> On 4 Apr 2017, at 18:37, jim holtman <[hidden email]> wrote:
>>
>> How big is 'large'?
>>
>> Jim Holtman
>> Data Munger Guru
>>
>> What is the problem that you are trying to solve?
>> Tell me what you want to do, not how you want to do it.
>>
>>
>> On Tue, Apr 4, 2017 at 7:47 AM, Louisa Reynolds via R-help
>> <[hidden email]> wrote:
>>> Dear Forum
>>> I am trying to cut out a small section of a very large 2-dimensional grayscale image as a tiff in R, but it is having difficulty handling such large files.  I have looked at bigmemory and ff packages but it is unclear how I can use these packages with tiffs. Can anyone please suggest something? I have tried tiff and rtiff libraries.
>>> Thanks in advance.
>>>        [[alternative HTML version deleted]]
>>>
>>> ______________________________________________
>>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: taking a small piece of large tiff

Ben Tupper-2
In reply to this post by R help mailing list-2
Hi,

You'll get a lot of help if you take this question to the R-sig-geo list

https://stat.ethz.ch/mailman/listinfo/r-sig-geo

In the meantime, you might want to try reading this using raster::raster() and then cropping to your desired region. Here's a discussion that may help you...

https://stat.ethz.ch/pipermail/r-sig-geo/2010-July/008838.html

You will find many other useful bits using Rseek.org - try http://rseek.org/?q=subset+large+geotif

Cheers,
Ben

> On Apr 5, 2017, at 3:23 AM, Louisa Reynolds via R-help <[hidden email]> wrote:
>
> Ok. I have a tiff of size over 2GB. It covers a sixth of the Earth's surface and I'm trying to cut a UK piece out of it. The tiff I start with seems to be too large for R to handle.
>
>
> Sent from my iPhone
>
>> On 4 Apr 2017, at 18:37, jim holtman <[hidden email]> wrote:
>>
>> How big is 'large'?
>>
>> Jim Holtman
>> Data Munger Guru
>>
>> What is the problem that you are trying to solve?
>> Tell me what you want to do, not how you want to do it.
>>
>>
>> On Tue, Apr 4, 2017 at 7:47 AM, Louisa Reynolds via R-help
>> <[hidden email]> wrote:
>>> Dear Forum
>>> I am trying to cut out a small section of a very large 2-dimensional grayscale image as a tiff in R, but it is having difficulty handling such large files.  I have looked at bigmemory and ff packages but it is unclear how I can use these packages with tiffs. Can anyone please suggest something? I have tried tiff and rtiff libraries.
>>> Thanks in advance.
>>>       [[alternative HTML version deleted]]
>>>
>>> ______________________________________________
>>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

Ben Tupper
Bigelow Laboratory for Ocean Sciences
60 Bigelow Drive, P.O. Box 380
East Boothbay, Maine 04544
http://www.bigelow.org

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: taking a small piece of large tiff

Michael Sumner-2
In reply to this post by jholtman
On Wed, 5 Apr 2017 at 20:13 jim holtman <[hidden email]> wrote:

> if you have 8GB of memory it should be easy to handle.
>
>
>
TIFF is a container format and may be compressed internally, and so could
expand out  as a matrix it might be very many times larger than the file
size implies.

(Follow Ben's advice and use raster, which in turn uses rgdal to read
TIFF/GeoTIFF. It's ridiculously good).


Something like

library(raster)
r <- raster("/path/to/myfile.tif")
print(r)  ## to see the units of the extent

## enter the singleton-values for xmin/xmax/ymin/ymax that you want here:
b <- crop(r, extent(xmin, xmax, ymin, ymax))

Note that the extent values must be in the coordinate system used by the
raster itself, it might be in longitude-latitude or in some
"eastings-northings" map projection (CRS, or coordinate reference system).
The print out of print(r) will tell you, and projection(r) if you need to
use it directly.

There's no need to do your own transformations from world-space to pixel
index space. but the price of that convenience is you have to interact with
the data via the raster package's design and interfaces. You can get a
matrix out but it's well worth learning the higher level abstractions
available as well.


Cheers, Mike.



> Jim Holtman
> Data Munger Guru
>
> What is the problem that you are trying to solve?
> Tell me what you want to do, not how you want to do it.
>
>
> On Wed, Apr 5, 2017 at 3:23 AM, Louisa Reynolds
> <[hidden email]> wrote:
> > Ok. I have a tiff of size over 2GB. It covers a sixth of the Earth's
> surface and I'm trying to cut a UK piece out of it. The tiff I start with
> seems to be too large for R to handle.
> >
> >
> > Sent from my iPhone
> >
> >> On 4 Apr 2017, at 18:37, jim holtman <[hidden email]> wrote:
> >>
> >> How big is 'large'?
> >>
> >> Jim Holtman
> >> Data Munger Guru
> >>
> >> What is the problem that you are trying to solve?
> >> Tell me what you want to do, not how you want to do it.
> >>
> >>
> >> On Tue, Apr 4, 2017 at 7:47 AM, Louisa Reynolds via R-help
> >> <[hidden email]> wrote:
> >>> Dear Forum
> >>> I am trying to cut out a small section of a very large 2-dimensional
> grayscale image as a tiff in R, but it is having difficulty handling such
> large files.  I have looked at bigmemory and ff packages but it is unclear
> how I can use these packages with tiffs. Can anyone please suggest
> something? I have tried tiff and rtiff libraries.
> >>> Thanks in advance.
> >>>        [[alternative HTML version deleted]]
> >>>
> >>> ______________________________________________
> >>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> >>> https://stat.ethz.ch/mailman/listinfo/r-help
> >>> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> >>> and provide commented, minimal, self-contained, reproducible code.
> >
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
--
Dr. Michael Sumner
Software and Database Engineer
Australian Antarctic Division
203 Channel Highway
Kingston Tasmania 7050 Australia

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: taking a small piece of large tiff

Ranjan Maitra-3
In reply to this post by R help mailing list-2
Hello Louisa,

THis is not a R solution but would it not be easier to use ImageMagick to do what you are wanting to do? Look up
https://www.imagemagick.org/script/index.php

HTH,
Ranjan

On Wed, 5 Apr 2017 08:23:51 +0100 Louisa Reynolds via R-help <[hidden email]> wrote:

> Ok. I have a tiff of size over 2GB. It covers a sixth of the Earth's surface and I'm trying to cut a UK piece out of it. The tiff I start with seems to be too large for R to handle.
>
>
> Sent from my iPhone
>
> > On 4 Apr 2017, at 18:37, jim holtman <[hidden email]> wrote:
> >
> > How big is 'large'?
> >
> > Jim Holtman
> > Data Munger Guru
> >
> > What is the problem that you are trying to solve?
> > Tell me what you want to do, not how you want to do it.
> >
> >
> > On Tue, Apr 4, 2017 at 7:47 AM, Louisa Reynolds via R-help
> > <[hidden email]> wrote:
> >> Dear Forum
> >> I am trying to cut out a small section of a very large 2-dimensional grayscale image as a tiff in R, but it is having difficulty handling such large files.  I have looked at bigmemory and ff packages but it is unclear how I can use these packages with tiffs. Can anyone please suggest something? I have tried tiff and rtiff libraries.
> >> Thanks in advance.
> >>        [[alternative HTML version deleted]]
> >>
> >> ______________________________________________
> >> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> >> and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>


--
Important Notice: This mailbox is ignored: e-mails are set to be deleted on receipt. Please respond to the mailing list if appropriate. For those needing to send personal or professional e-mail, please use appropriate addresses.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.