Hot Air Balloon Weather Briefings

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

Hot Air Balloon Weather Briefings

Philip
I’m trying to compare National Weather Service Rapid Update Forecast (RAP) data to GPS breadcrumbs collected by a really clever Apple Phone Ap that lays down longitude, latitude, altitude, compass direction, and speed every six seconds.   Below is a small subset of the GPS data from another flight.  

I want to delete the rows where the balloon does not move (Speed column) for a full minute assuming that it is sitting on the ground – beginning of the flight, changing passengers, or waiting for the chase crew at the end of the flight.  for example, I want to eliminate the data for minute 30 but keep the data for minute 31 because the balloon starts to move again at second 17.  Any suggestions?  I’ve tried putzing around with multiple lags without success.

      Minute Second Speed
      29 47 0
      29 53 0
      29 59 0
      30 5 0
      30 11 0
      30 17 0
      30 23 0
      30 29 0
      30 35 0
      30 41 0
      30 47 0
      30 53 0
      30 59 0
      31 5 0
      31 11 0
      31 17 0.402649
      31 23 0.671081
      31 29 1.588225
      31 35 2.438261
      31 41 2.706693
      31 47 2.930386
      31 53 3.310666
      31 59 3.198819
      32 5 3.422512


It would be even better if I could delete the rows where there were ten consecutive zero speed entries such as from minute 30 second 17 to minute 31 second 11.

Thanks,
Philip Heinrich
        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Hot Air Balloon Weather Briefings

Jim Lemon-4
Hi Philip,
Not very elegant, but:

phdf<-read.table(text="Minute Second Speed
 29 47 0
 29 53 0
 29 59 0
 30 5 0
 30 11 0
 30 17 0
 30 23 0
 30 29 0
 30 35 0
 30 41 0
 30 47 0
 30 53 0
 30 59 0
 31 5 0
 31 11 0
 31 17 0.402649
 31 23 0.671081
 31 29 1.588225
 31 35 2.438261
 31 41 2.706693
 31 47 2.930386
 31 53 3.310666
 31 59 3.198819
 32 5 3.422512",
 header=TRUE,stringsAsFactors=FALSE)
keep<-rep(TRUE,length(phdf$Speed))
for(mini in unique(phdf$Minute))
 if(all(phdf$Speed[phdf$Minute == mini] == 0))
  keep[phdf$Minute == mini]<-FALSE
phdf<-phdf[keep,]

Jim

On Sat, Aug 15, 2020 at 6:59 AM Philip <[hidden email]> wrote:

>
> I’m trying to compare National Weather Service Rapid Update Forecast (RAP) data to GPS breadcrumbs collected by a really clever Apple Phone Ap that lays down longitude, latitude, altitude, compass direction, and speed every six seconds.   Below is a small subset of the GPS data from another flight.
>
> I want to delete the rows where the balloon does not move (Speed column) for a full minute assuming that it is sitting on the ground – beginning of the flight, changing passengers, or waiting for the chase crew at the end of the flight.  for example, I want to eliminate the data for minute 30 but keep the data for minute 31 because the balloon starts to move again at second 17.  Any suggestions?  I’ve tried putzing around with multiple lags without success.
>
>       Minute Second Speed
>       29 47 0
>       29 53 0
>       29 59 0
>       30 5 0
>       30 11 0
>       30 17 0
>       30 23 0
>       30 29 0
>       30 35 0
>       30 41 0
>       30 47 0
>       30 53 0
>       30 59 0
>       31 5 0
>       31 11 0
>       31 17 0.402649
>       31 23 0.671081
>       31 29 1.588225
>       31 35 2.438261
>       31 41 2.706693
>       31 47 2.930386
>       31 53 3.310666
>       31 59 3.198819
>       32 5 3.422512
>
>
> It would be even better if I could delete the rows where there were ten consecutive zero speed entries such as from minute 30 second 17 to minute 31 second 11.
>
> Thanks,
> Philip Heinrich
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Hot Air Balloon Weather Briefings

Bert Gunter-2
In reply to this post by Philip
Well which is it?:
"I want to eliminate the data for minute 30 but keep the data for minute 31
because the balloon starts to move again at second 17. "
or
"It would be even better if I could delete the rows where there were ten
consecutive zero speed entries such as from minute 30 second 17 to minute
31 second 11."

If you want to delete say data with >= 10 consecutive 0's, ?rle is your
friend:

rle(phdf$Speed)
Run Length Encoding
  lengths: int [1:10] 15 1 1 1 1 1 1 1 1 1
  values : num [1:10] 0 0.402649 0.671081 1.588225 2.438261 2.706693
2.930386 3.310666 3.198819 3.422512

e.g.
> z <- rle(phdf$Speed)$lengths
## which gives:
> z
 [1] 15  1  1  1  1  1  1  1  1  1
> rep(z>9,z)
 [1]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
[11]  TRUE  TRUE  TRUE  TRUE  TRUE FALSE FALSE FALSE FALSE FALSE
[21] FALSE FALSE FALSE FALSE

##Thus:
> phdf[ rep(z<10, z), ]
   Minute Second    Speed
16     31     17 0.402649
17     31     23 0.671081
18     31     29 1.588225
19     31     35 2.438261
20     31     41 2.706693
21     31     47 2.930386
22     31     53 3.310666
23     31     59 3.198819
24     32      5 3.422512

But of course, this probably isn't what you meant. So further clarification
is needed.

Bert Gunter

"The trouble with having an open mind is that people keep coming along and
sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Fri, Aug 14, 2020 at 1:59 PM Philip <[hidden email]> wrote:

> I’m trying to compare National Weather Service Rapid Update Forecast (RAP)
> data to GPS breadcrumbs collected by a really clever Apple Phone Ap that
> lays down longitude, latitude, altitude, compass direction, and speed every
> six seconds.   Below is a small subset of the GPS data from another
> flight.
>
> I want to delete the rows where the balloon does not move (Speed column)
> for a full minute assuming that it is sitting on the ground – beginning of
> the flight, changing passengers, or waiting for the chase crew at the end
> of the flight.  for example, I want to eliminate the data for minute 30 but
> keep the data for minute 31 because the balloon starts to move again at
> second 17.  Any suggestions?  I’ve tried putzing around with multiple lags
> without success.
>
>       Minute Second Speed
>       29 47 0
>       29 53 0
>       29 59 0
>       30 5 0
>       30 11 0
>       30 17 0
>       30 23 0
>       30 29 0
>       30 35 0
>       30 41 0
>       30 47 0
>       30 53 0
>       30 59 0
>       31 5 0
>       31 11 0
>       31 17 0.402649
>       31 23 0.671081
>       31 29 1.588225
>       31 35 2.438261
>       31 41 2.706693
>       31 47 2.930386
>       31 53 3.310666
>       31 59 3.198819
>       32 5 3.422512
>
>
> It would be even better if I could delete the rows where there were ten
> consecutive zero speed entries such as from minute 30 second 17 to minute
> 31 second 11.
>
> Thanks,
> Philip Heinrich
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Hot Air Balloon Weather Briefings

Rasmus Liland-3
In reply to this post by Philip
On 2020-08-14 13:58 -0700, Philip wrote:
| I’m trying to compare National Weather
| Service Rapid Update Forecast (RAP)
| data to GPS breadcrumbs collected by a
| really clever Apple Phone Ap that lays
| down longitude, latitude, altitude,
| compass direction, and speed every six
| seconds.   Below is a small subset of
| the GPS data from another flight.  
|
| I want to delete the rows where the
| balloon does not move (Speed column)
| for a full minute assuming that it is
| sitting on the ground – beginning of
| the flight, changing passengers, or
| waiting for the chase crew at the end
| of the flight.  for example, I want to
| eliminate the data for minute 30 but
| keep the data for minute 31 because
| the balloon starts to move again at
| second 17.  Any suggestions?  I’ve
| tried putzing around with multiple
| lags without success.
|
| Minute Second Speed
| [...]
|
| It would be even better if I could
| delete the rows where there were ten
| consecutive zero speed entries such as
| from minute 30 second 17 to minute 31
| second 11.

Dear Philip,

first I though about solving this using
some combination of unique, duplicated,
table ... then I saw Jim's reply, and
rewrote it a little:

        rap <-
        structure(list(Minute = c(29L, 29L, 29L,
        30L, 30L, 30L, 30L, 30L, 30L, 30L, 30L,
        30L, 30L, 31L, 31L, 31L, 31L, 31L, 31L,
        31L, 31L, 31L, 31L, 31L, 31L, 31L, 31L,
        31L, 31L, 31L, 31L, 31L, 31L, 31L, 31L,
        31L, 31L, 31L, 31L, 31L, 31L, 32L),
        Second = c(47L, 53L, 59L, 5L, 11L, 17L,
        23L, 29L, 35L, 41L, 47L, 53L, 59L, 5L,
        11L, 17L, 23L, 29L, 35L, 41L, 43L, 43L,
        43L, 43L, 43L, 43L, 43L, 43L, 43L, 43L,
        43L, 43L, 47L, 53L, 54L, 54L, 54L, 54L,
        54L, 54L, 59L, 5L),
        Speed = c(0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
        0, 0, 0, 0, 0, 0.402649, 0.671081,
        1.588225, 2.438261, 2.706693, 0, 0, 0,
        0, 0, 0, 0, 0, 0, 0, 0, 0, 2.930386,
        3.310666, 0, 0, 0, 0, 0, 0, 3.198819,
        3.422512)), class = "data.frame",
        row.names = c(NA, -42L))
       
        minis <- unique(rap$Minute)
        FUN <- function(mini, rap) {
          all(rap$Speed[rap$Minute==mini]==0) }
        keep <- rap$Minute %in%
          minis[!simplify2array(
            parallel::mclapply(minis, FUN, rap))]
        rap[keep,]

As to Bert's reply, I am a loss as how
to use the lengths list in
rle(rap$Speed) for this ...

Best,
Rasmus

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Hot Air Balloon Weather Briefings

Rasmus Liland-3
On 2020-08-14 15:56 -0700, Bert Gunter wrote:
| On Fri, Aug 14, 2020 at 3:21 PM Rasmus Liland wrote:
| |
| | As to Bert's reply, I am a loss as
| | how to use the lengths list in
| | rle(rap$Speed) for this ...
|
| I showed how in my message for one
| interpretation of the query. I would
| need further clarification for other
| interpretations, but note that in my
| reply,   rep(z|9, z)  gives a logical
| vector in which all rows in which a
| speed of 0 appears in a run of 10 or
| greater (as no other speed would be
| replicated) is TRUE, FALSE otherwise.  
| ( z = $lengths).  I would agree that
| whether this is a good starting point
| for other interpretations remains to
| be seen.

Yes, the time cols might be omitted to
complete this.

        z <- rle(rap$Speed)$lengths
        rap[rep(x=z<10, times=z),]

This is not bad.

/Rasmus

P.S. Adding this back to the list (this
ends up in a searchable archive, and so
on and so fourth).

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

signature.asc (849 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Hot Air Balloon Weather Briefings

Philip
In reply to this post by Bert Gunter-2
Your suggestion worked like a charm.  Thank you

rle <- rle(TF$Speed)$lengths #Counts number of repetitions
phdf$rle <- rep(rle>9,rle) #TRUE is ten or more zeros.  FALSE if less than 10
phdf2 <- TF[rep(rle<10,rle),] #Move rows that are FALSE to new data file

From: Bert Gunter
Sent: Friday, August 14, 2020 2:43 PM
To: Philip
Cc: r-help
Subject: Re: [R] Hot Air Balloon Weather Briefings

Well which is it?:
"I want to eliminate the data for minute 30 but keep the data for minute 31 because the balloon starts to move again at second 17. "
or
"It would be even better if I could delete the rows where there were ten consecutive zero speed entries such as from minute 30 second 17 to minute 31 second 11."

If you want to delete say data with >= 10 consecutive 0's, ?rle is your friend:

rle(phdf$Speed)
Run Length Encoding
  lengths: int [1:10] 15 1 1 1 1 1 1 1 1 1
  values : num [1:10] 0 0.402649 0.671081 1.588225 2.438261 2.706693 2.930386 3.310666 3.198819 3.422512

e.g.

> z <- rle(phdf$Speed)$lengths
## which gives:

> z
[1] 15  1  1  1  1  1  1  1  1  1
> rep(z>9,z)
[1]  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE  TRUE
[11]  TRUE  TRUE  TRUE  TRUE  TRUE FALSE FALSE FALSE FALSE FALSE
[21] FALSE FALSE FALSE FALSE

##Thus:

> phdf[ rep(z<10, z), ]
   Minute Second    Speed
16     31     17 0.402649
17     31     23 0.671081
18     31     29 1.588225
19     31     35 2.438261
20     31     41 2.706693
21     31     47 2.930386
22     31     53 3.310666
23     31     59 3.198819
24     32      5 3.422512

But of course, this probably isn't what you meant. So further clarification is needed.

Bert Gunter

"The trouble with having an open mind is that people keep coming along and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )



On Fri, Aug 14, 2020 at 1:59 PM Philip <[hidden email]> wrote:

  I’m trying to compare National Weather Service Rapid Update Forecast (RAP) data to GPS breadcrumbs collected by a really clever Apple Phone Ap that lays down longitude, latitude, altitude, compass direction, and speed every six seconds.   Below is a small subset of the GPS data from another flight.  

  I want to delete the rows where the balloon does not move (Speed column) for a full minute assuming that it is sitting on the ground – beginning of the flight, changing passengers, or waiting for the chase crew at the end of the flight.  for example, I want to eliminate the data for minute 30 but keep the data for minute 31 because the balloon starts to move again at second 17.  Any suggestions?  I’ve tried putzing around with multiple lags without success.

        Minute Second Speed
        29 47 0
        29 53 0
        29 59 0
        30 5 0
        30 11 0
        30 17 0
        30 23 0
        30 29 0
        30 35 0
        30 41 0
        30 47 0
        30 53 0
        30 59 0
        31 5 0
        31 11 0
        31 17 0.402649
        31 23 0.671081
        31 29 1.588225
        31 35 2.438261
        31 41 2.706693
        31 47 2.930386
        31 53 3.310666
        31 59 3.198819
        32 5 3.422512


  It would be even better if I could delete the rows where there were ten consecutive zero speed entries such as from minute 30 second 17 to minute 31 second 11.

  Thanks,
  Philip Heinrich
          [[alternative HTML version deleted]]

  ______________________________________________
  [hidden email] mailing list -- To UNSUBSCRIBE and more, see
  https://stat.ethz.ch/mailman/listinfo/r-help
  PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
  and provide commented, minimal, self-contained, reproducible code.

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Hot Air Balloon Weather Briefings

Jim Lemon-4
In reply to this post by Jim Lemon-4

Hi Philip,
My fault for assuming that what worked for the sample data would work
for the entire data set. If you run the following code:

# read the file into a data frame
phdf<-read.csv("phdf.csv",stringsAsFactors=FALSE)
print(dim(phdf))
# create a logical variable for the subsetting step
keep<-rep(TRUE,length(phdf$Speed))
# this follows the conventional for(i in ...) syntax
# mini (minute index) is the name of the variable
# that is assigned the successive values of the
# unique values in the Minute column of phdf
# here I was lazy and only dealt with the sample data
# what I should have done was to create a column
# with minute values that don't repeat
phdf$hourmin<-phdf$Hour.Z * 60 + phdf$Minute
for(mini in unique(phdf$hourmin)) {
 # mark all minutes that are all zeros for deletion
 if(all(phdf$Speed[phdf$hourmin == mini] == 0))
  keep[phdf$hourmin == mini]<-FALSE
 # but now there is another condition
 # that I didn't notice (more laziness)
 # drop minutes containing ten consecutive zeros
 # I'll use a cheap trick for this
 # if the length of the run length encoding (rle)
 # of the Speed == 0 condition is less than three,
 # then there can't be a broken run of zeros
 if(length(rle(phdf$Speed[phdf$hourmin == mini])$length)<3)
  keep[phdf$hourmin == mini]<-FALSE
}
# now drop any rows for which has marked FALSE (note all caps!)
phdf<-phdf[keep,]
print(dim(phdf))

You will see that it has removed 67 rows. This is the same as if I had
only applied the first "all zeros"  condition, for there were no
unique minutes with 11 observations that contained a single non-zero
speed value. I can see that you are getting short runs of zero speeds
near the end. I assume that this is due to the balloon slowly bumping
along the ground. Often just looking at the data can suggest solutions
to problems like this.

Coincidentally I am about to email an old friend of mine whose sons
have dabbled in sending balloons high into the air and I will let them
know that they are not alone in performing this unusual practice. If I
haven't answered all your questions, feel free to let me know.

Jim

On Sun, Aug 16, 2020 at 4:39 AM Philip <[hidden email]> wrote:

>
> Thanks for getting back to me so quickly.
>
> I can get your code to run without errors but I'm not sure what it
> accomplishes since I still get 813 rows of data and 11 variables.  The
> entire file for a January flight is attached.  Also attached is a .jpg of a
> flight from a couple of years ago where my wife putzed back and forth across
> a road for over an hour by going up or down to catch different winds.  Stuff
> like this is one of the charms of the sport for those of us who are easily
> amused.
>
> keep <- rep(TRUE,length(phdf$Speed))             #813 repetitions of TRUE
> for(mini in unique(phdf$Minute))
>   if(all(phdf$Speed[phdf$Minute==mini]==0))
>   keep[phdf$Minute==mini]<-False                     #813 repetitions of
> TRUE in the keep data file.
>
> #Don't understand the assignment (<-) to FALSE
> phdf <- phdf[keep,] #Still have 813 rows of data.
>
> As you may know, we lay out the balloon and then blow cold air into the
> envelope with a gas powered fan.  When it is packed we "go hot".  Just
> before we turn on the pilot light and hit the burner we will turn on the
> tracking software which results in several minutes of no movement at the
> beginning of the flight.  This is what is happening between rows 2 and 70 of
> the attached spreadsheet.  You will also notice that the balloon slowly
> accelerates between rows 71 and 79 p to about 3.4 MPH just after liftoff.
> The no movement is what I want to eliminate.
>
> Can you let me know what I am missing.
>
> Two more thing.  I really not sure about the:
>
>     for(mini in unique(phdf$Minute))
>
> .....line of code.  I understand the unique function but not the "mini
> in..." part.  Is mini a function or just a label?   I understand stuff like:
>
>     for(i in 1:5) print(1:i)
>
> .....from the R base documentation but not real sure how it fits in here.
>
> And finally, I'm retired,  So I have plenty of time and determined to learn
> R.  But I keep running into things like the "mini in unique" command.  I
> have read four or five books and watched or read dozens of tutorials but
> there always seems to be another layer that alludes me.  Any suggestions?
>
> Philip

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.