on execution time of a function...

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

on execution time of a function...

akshay kulkarni
I ran  a function in R three days ago and the execution time was about 4 minutes. I ran the same function yesterday and the execution time was more than 6:50 minutes(I aborted the function after that time).

I read in the Internet that this is possible. I also came to know that software or hardware interrupts are the main reasons.

How do you know whether the delay was caused by interrupts? Which hardware or software triggered the interrupts? In general, how to know the exact cause of the delay in execution in R? Are there any packages for these analyses?

Very many thanks for your time and effort....

yours sincerely,
AKSHAY M KULKARNI

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: on execution time of a function...

Jeff Newmiller
Wow, you can find almost any explanation on the Internets. That doesn't mean you should believe all of them. R does not do anything likely to tweak interrupts... if that is your problem then you need to be on an operating-system/computer-model-specific forum rather than this OS-agnostic mailing list.

It is far more likely that your overall memory usage conditions have changed since the last time you ran it... or that you didn't actually record all of the things you did last time in your script. (Newbie R users often do things at the console without putting them in their scripts.)

I suggest that you run your script one statement at a time and see where your problem is. You might also want to make sure that other programs are not using up a lot of your memory (which could involve some OS-specific tools or just shutting down some other programs.)

On June 12, 2018 7:36:40 PM HST, akshay kulkarni <[hidden email]> wrote:

>I ran  a function in R three days ago and the execution time was about
>4 minutes. I ran the same function yesterday and the execution time was
>more than 6:50 minutes(I aborted the function after that time).
>
>I read in the Internet that this is possible. I also came to know that
>software or hardware interrupts are the main reasons.
>
>How do you know whether the delay was caused by interrupts? Which
>hardware or software triggered the interrupts? In general, how to know
>the exact cause of the delay in execution in R? Are there any packages
>for these analyses?
>
>Very many thanks for your time and effort....
>
>yours sincerely,
>AKSHAY M KULKARNI
>
> [[alternative HTML version deleted]]
>
>______________________________________________
>[hidden email] mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

--
Sent from my phone. Please excuse my brevity.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: on execution time of a function...

Loris Bennett-2
Hi Akshay,

In addition to all the things Jeff rightly points out, contention for IO
resources can be an issue.  So if another process was hogging the
bandwidth while your program was attempting to read or write to disk,
that could also have slowed things down.

HTH

Loris

Jeff Newmiller <[hidden email]> writes:

> Wow, you can find almost any explanation on the Internets. That doesn't mean you
> should believe all of them. R does not do anything likely to tweak
> interrupts... if that is your problem then you need to be on an
> operating-system/computer-model-specific forum rather than this OS-agnostic
> mailing list.
>
> It is far more likely that your overall memory usage conditions have changed
> since the last time you ran it... or that you didn't actually record all of the
> things you did last time in your script. (Newbie R users often do things at the
> console without putting them in their scripts.)
>
> I suggest that you run your script one statement at a time and see where your
> problem is. You might also want to make sure that other programs are not using
> up a lot of your memory (which could involve some OS-specific tools or just
> shutting down some other programs.)
>
> On June 12, 2018 7:36:40 PM HST, akshay kulkarni <[hidden email]> wrote:
>>I ran  a function in R three days ago and the execution time was about
>>4 minutes. I ran the same function yesterday and the execution time was
>>more than 6:50 minutes(I aborted the function after that time).
>>
>>I read in the Internet that this is possible. I also came to know that
>>software or hardware interrupts are the main reasons.
>>
>>How do you know whether the delay was caused by interrupts? Which
>>hardware or software triggered the interrupts? In general, how to know
>>the exact cause of the delay in execution in R? Are there any packages
>>for these analyses?
>>
>>Very many thanks for your time and effort....
>>
>>yours sincerely,
>>AKSHAY M KULKARNI
>>
>> [[alternative HTML version deleted]]
>>
>>______________________________________________
>>[hidden email] mailing list -- To UNSUBSCRIBE and more, see
>>https://stat.ethz.ch/mailman/listinfo/r-help
>>PLEASE do read the posting guide
>>http://www.R-project.org/posting-guide.html
>>and provide commented, minimal, self-contained, reproducible code.
--
Dr. Loris Bennett (Mr.)
ZEDAT, Freie Universität Berlin         Email [hidden email]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: on execution time of a function...

akshay kulkarni
I do felt a little unsettled by your exhortions on the incongruity of posting this question on an OS-agnostic mailing list...I thought that there might be some issues on how R communicates with the OS...and also that some R packages might rectify the issue(in my experience, I have had a R package for every R issue that I had!)

Anyway, I ran the  same function again and it is executing within limits. Also, I would be using AWS EC2 servers to run my R functions(I am a day trader in india and input some 250 stocks to R functions daily), and I don't think that the issue would persist on Intel Xeon processors and dedicated VMs....

Anyway,thanks for all your concerns tolerating my query on an OS-agnostic mailing list!

AKSHAY M KULKARNI
________________________________
From: Loris Bennett <[hidden email]>
Sent: Wednesday, June 13, 2018 1:37 PM
To: Jeff Newmiller
Cc: [hidden email]; akshay kulkarni
Subject: Re: [R] on execution time of a function...

Hi Akshay,

In addition to all the things Jeff rightly points out, contention for IO
resources can be an issue.  So if another process was hogging the
bandwidth while your program was attempting to read or write to disk,
that could also have slowed things down.

HTH

Loris

Jeff Newmiller <[hidden email]> writes:

> Wow, you can find almost any explanation on the Internets. That doesn't mean you
> should believe all of them. R does not do anything likely to tweak
> interrupts... if that is your problem then you need to be on an
> operating-system/computer-model-specific forum rather than this OS-agnostic
> mailing list.
>
> It is far more likely that your overall memory usage conditions have changed
> since the last time you ran it... or that you didn't actually record all of the
> things you did last time in your script. (Newbie R users often do things at the
> console without putting them in their scripts.)
>
> I suggest that you run your script one statement at a time and see where your
> problem is. You might also want to make sure that other programs are not using
> up a lot of your memory (which could involve some OS-specific tools or just
> shutting down some other programs.)
>
> On June 12, 2018 7:36:40 PM HST, akshay kulkarni <[hidden email]> wrote:
>>I ran  a function in R three days ago and the execution time was about
>>4 minutes. I ran the same function yesterday and the execution time was
>>more than 6:50 minutes(I aborted the function after that time).
>>
>>I read in the Internet that this is possible. I also came to know that
>>software or hardware interrupts are the main reasons.
>>
>>How do you know whether the delay was caused by interrupts? Which
>>hardware or software triggered the interrupts? In general, how to know
>>the exact cause of the delay in execution in R? Are there any packages
>>for these analyses?
>>
>>Very many thanks for your time and effort....
>>
>>yours sincerely,
>>AKSHAY M KULKARNI
>>
>>      [[alternative HTML version deleted]]
>>
>>______________________________________________
>>[hidden email] mailing list -- To UNSUBSCRIBE and more, see
>>https://stat.ethz.ch/mailman/listinfo/r-help
>>PLEASE do read the posting guide
>>http://www.R-project.org/posting-guide.html
>>and provide commented, minimal, self-contained, reproducible code.
--
Dr. Loris Bennett (Mr.)
ZEDAT, Freie Universit�t Berlin         Email [hidden email]

        [[alternative HTML version deleted]]


______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: on execution time of a function...

Jeff Newmiller
it is not just the list that is OS-agnostic... R itself is quite OS-agnostic. However, depending on where you get your packages from they may have something unusual going on in their compiled C or Fortran code. It is just that for a computationally-oriented application like R code that messed with hardware interrupts seem out of place.

I suppose that a bug in a CUDA-related package could screw up interrupts... but the whole point of having an OS as hardware drivers/APIs is to insulate user-level programs like R from those issues, and at best you would post the details of your code and packages and someone might pick up on an odd package and point you toward a more appropriate forum for debugging it.

On June 12, 2018 11:39:40 PM HST, akshay kulkarni <[hidden email]> wrote:

>I do felt a little unsettled by your exhortions on the incongruity of
>posting this question on an OS-agnostic mailing list...I thought that
>there might be some issues on how R communicates with the OS...and also
>that some R packages might rectify the issue(in my experience, I have
>had a R package for every R issue that I had!)
>
>Anyway, I ran the  same function again and it is executing within
>limits. Also, I would be using AWS EC2 servers to run my R functions(I
>am a day trader in india and input some 250 stocks to R functions
>daily), and I don't think that the issue would persist on Intel Xeon
>processors and dedicated VMs....
>
>Anyway,thanks for all your concerns tolerating my query on an
>OS-agnostic mailing list!
>
>AKSHAY M KULKARNI
>________________________________
>From: Loris Bennett <[hidden email]>
>Sent: Wednesday, June 13, 2018 1:37 PM
>To: Jeff Newmiller
>Cc: [hidden email]; akshay kulkarni
>Subject: Re: [R] on execution time of a function...
>
>Hi Akshay,
>
>In addition to all the things Jeff rightly points out, contention for
>IO
>resources can be an issue.  So if another process was hogging the
>bandwidth while your program was attempting to read or write to disk,
>that could also have slowed things down.
>
>HTH
>
>Loris
>
>Jeff Newmiller <[hidden email]> writes:
>
>> Wow, you can find almost any explanation on the Internets. That
>doesn't mean you
>> should believe all of them. R does not do anything likely to tweak
>> interrupts... if that is your problem then you need to be on an
>> operating-system/computer-model-specific forum rather than this
>OS-agnostic
>> mailing list.
>>
>> It is far more likely that your overall memory usage conditions have
>changed
>> since the last time you ran it... or that you didn't actually record
>all of the
>> things you did last time in your script. (Newbie R users often do
>things at the
>> console without putting them in their scripts.)
>>
>> I suggest that you run your script one statement at a time and see
>where your
>> problem is. You might also want to make sure that other programs are
>not using
>> up a lot of your memory (which could involve some OS-specific tools
>or just
>> shutting down some other programs.)
>>
>> On June 12, 2018 7:36:40 PM HST, akshay kulkarni
><[hidden email]> wrote:
>>>I ran  a function in R three days ago and the execution time was
>about
>>>4 minutes. I ran the same function yesterday and the execution time
>was
>>>more than 6:50 minutes(I aborted the function after that time).
>>>
>>>I read in the Internet that this is possible. I also came to know
>that
>>>software or hardware interrupts are the main reasons.
>>>
>>>How do you know whether the delay was caused by interrupts? Which
>>>hardware or software triggered the interrupts? In general, how to
>know
>>>the exact cause of the delay in execution in R? Are there any
>packages
>>>for these analyses?
>>>
>>>Very many thanks for your time and effort....
>>>
>>>yours sincerely,
>>>AKSHAY M KULKARNI
>>>
>>>      [[alternative HTML version deleted]]
>>>
>>>______________________________________________
>>>[hidden email] mailing list -- To UNSUBSCRIBE and more, see
>>>https://stat.ethz.ch/mailman/listinfo/r-help
>>>PLEASE do read the posting guide
>>>http://www.R-project.org/posting-guide.html
>>>and provide commented, minimal, self-contained, reproducible code.
>--
>Dr. Loris Bennett (Mr.)
>ZEDAT, Freie Universität Berlin         Email
>[hidden email]

--
Sent from my phone. Please excuse my brevity.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.