R Timeseries tsoutliers:tso

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

R Timeseries tsoutliers:tso

Darshan Pandya
Hello,

Writing to seek help in regard to some unexpected performance anomaly i am
observing in using tsoutlers:tso on the mac vs on an AWS cloud server..


I am running the following code with very small dataset of about 208
records.

d.dir <- '/Users/darshanpandya/xxxxxx'
FNAME <- 'my_data.csv'

d.input <- fread(file.path(paste0(d.dir,"/zzz/"),FNAME,fsep =
.Platform$file.sep), header = TRUE, stringsAsFactors = FALSE, showProgress
= TRUE) %>% as.data.frame
d.input$dummy_date <- as.Date(d.input$dummy_date, "%Y-%m-%d")

selectedCols <- c("observation","dummy_date")
inputData <- d.input[,selectedCols]
outlier.types <- list("Additive Outliers","Levels Shifts","Seasonal Level
Shifts","Innovation Outlier","Temporary Change")
tsFrequency <- 52
params <- list("outlier.types" = outlier.types,"tsFrequency" = tsFrequency)

tsFrequency = params$tsFrequency
outTypes <- unlist(params$outlier.types)
ts.var <- ts(inputData[,1], frequency = tsFrequency)
ts_outlier <- tso(ts.var, types =
​outlier.types ​
, maxit = 1)



This piece of code runs in about 60 seconds on my laptop but take about
upward of 13 mins on a decent AWS node with nothing else running on it.

Any hints as to why this is happening is greatly appreciated.
Thank you for your help.


--
Sincerely,
Darshan

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Fwd: R Timeseries tsoutliers:tso

Darshan Pandya
Hello,

Writing to seek help in regard to some unexpected performance anomaly i am
observing in using tsoutlers:tso on the mac vs on an AWS cloud server..


I am running the following code with very small dataset of about 208
records.

d.dir <- '/Users/darshanpandya/xxxxxx'
FNAME <- 'my_data.csv'

d.input <- fread(file.path(paste0(d.dir,"/zzz/"),FNAME,fsep =
.Platform$file.sep), header = TRUE, stringsAsFactors = FALSE, showProgress
= TRUE) %>% as.data.frame
d.input$dummy_date <- as.Date(d.input$dummy_date, "%Y-%m-%d")

selectedCols <- c("observation","dummy_date")
inputData <- d.input[,selectedCols]
outlier.types <- list("Additive Outliers","Levels Shifts","Seasonal Level
Shifts","Innovation Outlier","Temporary Change")
tsFrequency <- 52
params <- list("outlier.types" = outlier.types,"tsFrequency" = tsFrequency)

tsFrequency = params$tsFrequency
outTypes <- unlist(params$outlier.types)
ts.var <- ts(inputData[,1], frequency = tsFrequency)
ts_outlier <- tso(ts.var, types =
​outlier.types ​
, maxit = 1)



This piece of code runs in about 60 seconds on my laptop but take about
upward of 13 mins on a decent AWS node with nothing else running on it.

Any hints as to why this is happening is greatly appreciated.
Thank you for your help.





--
Sincerely,
Darshan

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Fwd: R Timeseries tsoutliers:tso

R help mailing list-2
You can record the time to evaluate each line by wrapping each line in a
call to system.time().  E.g.,

expressions <- quote({
    # paste your commands here, or put them into a file and use exprs <-
parse("thatFile")
    d.dir <- '/Users/darshanpandya/xxxxxx'
FNAME <- 'my_data.csv'

d.input <- fread(file.path(paste0(d.dir,"/zzz/"),FNAME,fsep =
.Platform$file.sep), header = TRUE, stringsAsFactors = FALSE, showProgress
= TRUE) %>% as.data.frame
  # ... the rest of your lines
})
expressions <- as.list(expressions)[-1]

timedExpressions <- lapply(expressions,
function(expr)bquote(system.time(.(expr))))

source(exprs=timedExpressions, echo=TRUE)

See which line (or lines) show big time differences on the two platforms.


Bill Dunlap
TIBCO Software
wdunlap tibco.com

On Fri, Apr 13, 2018 at 8:33 AM, Darshan Pandya <[hidden email]>
wrote:

> Hello,
>
> Writing to seek help in regard to some unexpected performance anomaly i am
> observing in using tsoutlers:tso on the mac vs on an AWS cloud server..
>
>
> I am running the following code with very small dataset of about 208
> records.
>
> d.dir <- '/Users/darshanpandya/xxxxxx'
> FNAME <- 'my_data.csv'
>
> d.input <- fread(file.path(paste0(d.dir,"/zzz/"),FNAME,fsep =
> .Platform$file.sep), header = TRUE, stringsAsFactors = FALSE, showProgress
> = TRUE) %>% as.data.frame
> d.input$dummy_date <- as.Date(d.input$dummy_date, "%Y-%m-%d")
>
> selectedCols <- c("observation","dummy_date")
> inputData <- d.input[,selectedCols]
> outlier.types <- list("Additive Outliers","Levels Shifts","Seasonal Level
> Shifts","Innovation Outlier","Temporary Change")
> tsFrequency <- 52
> params <- list("outlier.types" = outlier.types,"tsFrequency" = tsFrequency)
>
> tsFrequency = params$tsFrequency
> outTypes <- unlist(params$outlier.types)
> ts.var <- ts(inputData[,1], frequency = tsFrequency)
> ts_outlier <- tso(ts.var, types =
> ​outlier.types ​
> , maxit = 1)
>
> ​
>
> This piece of code runs in about 60 seconds on my laptop but take about
> upward of 13 mins on a decent AWS node with nothing else running on it.
>
> Any hints as to why this is happening is greatly appreciated.
> Thank you for your help.
>
>
>
>
>
> --
> Sincerely,
> Darshan
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.