Tobit Fixed Effects

classic Classic list List threaded Threaded
24 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Tobit Fixed Effects

felipnunes
Hi there,

I need to run a Tobit Fixed Effects in a panel data with 4500 units for 8
years. It is a huge data set, my dependent variable is left truncated at
zero, the distribution is skewed and my panel is balanced.

Any suggestions on how to do that in R?

I tried stuff like survreg, censReg, and tobit but none of them were
satisfactory.

Thanks,

*Felipe Nunes*
CAPES/Fulbright Fellow
PhD Student Political Science - UCLA
Web: felipenunes.bol.ucla.edu

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Tobit Fixed Effects

Daniel Malter
Not satisfactory in which sense?

The survreg(Surv(Value,Censoring)~indvars+strata(id)) should/may work. For a discussion of Tobit fixed effects, see also Greene's website:

http://pages.stern.nyu.edu/~wgreene/publications.htm under "Fixed Efects and Bias Due to the Incidental Parameters Problem in the Tobit Model"

HTH,
Daniel
Reply | Threaded
Open this post in threaded view
|

Re: Tobit Fixed Effects

Joshua Wiley-2
On Wed, Sep 14, 2011 at 11:56 PM, Daniel Malter <[hidden email]> wrote:
> Not satisfactory in which sense?
>
> The survreg(Surv(Value,Censoring)~indvars+strata(id)) should/may work. For a

or cluster(year) if you have a new sample of people in each year

> discussion of Tobit fixed effects, see also Greene's website:
>
> http://pages.stern.nyu.edu/~wgreene/publications.htm under "Fixed Efects and
> Bias Due to the Incidental Parameters Problem in the Tobit Model"
>
> HTH,
> Daniel
>
> --
> View this message in context: http://r.789695.n4.nabble.com/Tobit-Fixed-Effects-tp3814771p3814830.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

--
Joshua Wiley
Ph.D. Student, Health Psychology
Programmer Analyst II, ATS Statistical Consulting Group
University of California, Los Angeles
https://joshuawiley.com/

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Tobit Fixed Effects

felipnunes
Let me explain my problem: I have a data set of municipalities from 2002 to
2010. For each municipality I have the amount of money per capita invested
in infra-structure for each year. This variable is truncated at zero, as
there is no negative values for investment and there are many cases in which
the municipality didn't receive any benefit. At the same time, the
investment per capita in some municipalities is really huge, making the
distribution very skewed.

My main independent variable is categorical (k=6). I, then, estimate my
models with 5 dummies, excluding one of the categories as reference. I also
have other continuous variables that I use for control purposes, such as GDP
capita and poverty rate.

Given all that, I'm trying to fit a tobit model that includes year and
municipalities fixed effects.

My technical problems were:

(1) tob3 <- censReg(transfers.cap ~ factor(mayor) + vot.weight + vot.per +
transfers.cap.lag + pib.cap + cluster(factor(year)), left=0, data = pool)

Using 'censReg' my results never converge. Std. errors are not calculated
and the coefficients are weird.

(2) when I tried

tob4 <- survreg(Surv(transfers.cap, trans.dummy) ~ factor(uf.mun) +
factor(mayor) + transfers.cap.lag + pib.cap + cluster(factor(year)),
dist='exponential', data=pool)

I got the following message

Error: cannot allocate vector of size 1.8 Gb
R(2080,0xa0437540) malloc: *** mmap(size=1885925376) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
R(2080,0xa0437540) malloc: *** mmap(size=1885925376) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug

(3) and I couldn't find the proper way to calculate my model using the tobit
function.

I appreciate your help!


*Felipe Nunes*
CAPES/Fulbright Fellow
PhD Student Political Science - UCLA
Web: felipenunes.bol.ucla.edu



On Thu, Sep 15, 2011 at 12:03 AM, Joshua Wiley <[hidden email]>wrote:

> On Wed, Sep 14, 2011 at 11:56 PM, Daniel Malter <[hidden email]> wrote:
> > Not satisfactory in which sense?
> >
> > The survreg(Surv(Value,Censoring)~indvars+strata(id)) should/may work.
> For a
>
> or cluster(year) if you have a new sample of people in each year
>
> > discussion of Tobit fixed effects, see also Greene's website:
> >
> > http://pages.stern.nyu.edu/~wgreene/publications.htm under "Fixed Efects
> and
> > Bias Due to the Incidental Parameters Problem in the Tobit Model"
> >
> > HTH,
> > Daniel
> >
> > --
> > View this message in context:
> http://r.789695.n4.nabble.com/Tobit-Fixed-Effects-tp3814771p3814830.html
> > Sent from the R help mailing list archive at Nabble.com.
> >
> > ______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
> --
> Joshua Wiley
> Ph.D. Student, Health Psychology
> Programmer Analyst II, ATS Statistical Consulting Group
> University of California, Los Angeles
> https://joshuawiley.com/
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Tobit Fixed Effects

Arne Henningsen-4
Dear Felipe

On 15 September 2011 09:58, Felipe Nunes <[hidden email]> wrote:
> My technical problems were:
>
> (1) tob3 <- censReg(transfers.cap ~ factor(mayor) + vot.weight + vot.per +
> transfers.cap.lag + pib.cap + cluster(factor(year)), left=0, data = pool)
>
> Using 'censReg' my results never converge. Std. errors are not calculated
> and the coefficients are weird.

Did you estimate the model for the pooled data set (i.e. not
considering the panel structure)? Or did you create a "panel data
frame" using pdata.frame(), which is available in package "plm", and
estimate a random-effects model?

What do you exactly mean with "never converge"? Did you try to
increase the maximum number of iterations (argument "iterlim")? Did
you try to use other optimization methods, e.g. BHHH (argument
"method")?

What is the effect of using "cluster()" in the model (formula) specification?

/Arne

--
Arne Henningsen
http://www.arne-henningsen.name

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Tobit Fixed Effects

felipnunes
Hi Arne,

I did both ways (pooled and pdata.frame), but in none I got a result. The
coefficients are estimated, but not the std. errors. I'm using BFGS method,
but I didn't increase the number of iterations yet. Let me try!

Best,

*Felipe Nunes*
CAPES/Fulbright Fellow
PhD Student Political Science - UCLA
Web: felipenunes.bol.ucla.edu



On Thu, Sep 15, 2011 at 3:33 AM, Arne Henningsen <
[hidden email]> wrote:

> Dear Felipe
>
> On 15 September 2011 09:58, Felipe Nunes <[hidden email]> wrote:
> > My technical problems were:
> >
> > (1) tob3 <- censReg(transfers.cap ~ factor(mayor) + vot.weight + vot.per
> +
> > transfers.cap.lag + pib.cap + cluster(factor(year)), left=0, data = pool)
> >
> > Using 'censReg' my results never converge. Std. errors are not calculated
> > and the coefficients are weird.
>
> Did you estimate the model for the pooled data set (i.e. not
> considering the panel structure)? Or did you create a "panel data
> frame" using pdata.frame(), which is available in package "plm", and
> estimate a random-effects model?
>
> What do you exactly mean with "never converge"? Did you try to
> increase the maximum number of iterations (argument "iterlim")? Did
> you try to use other optimization methods, e.g. BHHH (argument
> "method")?
>
> What is the effect of using "cluster()" in the model (formula)
> specification?
>
> /Arne
>
> --
> Arne Henningsen
> http://www.arne-henningsen.name
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Tobit Fixed Effects

Arne Henningsen-4
On 15 September 2011 20:09, Felipe Nunes <[hidden email]> wrote:
> I did both ways (pooled and pdata.frame), but in none I got a result. The
> coefficients are estimated, but not the std. errors. I'm using BFGS method,
> but I didn't increase the number of iterations yet. Let me try!

... and I highly recommend to use the BHHH method, particularly for
the random effects panel data estimation.

/Arne

--
Arne Henningsen
http://www.arne-henningsen.name

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Tobit Fixed Effects

felipnunes
Do you guys know how could I increase the time limit for the iterations in
censReg? I'm receiving the following message whatever method I use:

Newton-Raphson maximisation, 150 iterations
Return code 4: Iteration limit exceeded.
Log-likelihood: -67680.41 on 42 Df

thanks,

*Felipe Nunes*
CAPES/Fulbright Fellow
PhD Student Political Science - UCLA
Web: felipenunes.bol.ucla.edu



On Thu, Sep 15, 2011 at 11:40 AM, Arne Henningsen <
[hidden email]> wrote:

> On 15 September 2011 20:09, Felipe Nunes <[hidden email]> wrote:
> > I did both ways (pooled and pdata.frame), but in none I got a result. The
> > coefficients are estimated, but not the std. errors. I'm using BFGS
> method,
> > but I didn't increase the number of iterations yet. Let me try!
>
> ... and I highly recommend to use the BHHH method, particularly for
> the random effects panel data estimation.
>
> /Arne
>
> --
> Arne Henningsen
> http://www.arne-henningsen.name
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Tobit Fixed Effects

Arne Henningsen-4
On 18 September 2011 04:58, Felipe Nunes <[hidden email]> wrote:
> Do you guys know how could I increase the time limit for the iterations in
> censReg? I'm receiving the following message whatever method I use:
> Newton-Raphson maximisation, 150 iterations
> Return code 4: Iteration limit exceeded.
> Log-likelihood: -67680.41 on 42 Df

You can use argument "iterlim" to increase the maximum number of
iterations, see documentation of maxNR().

/Arne

--
Arne Henningsen
http://www.arne-henningsen.name

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Tobit Fixed Effects

felipnunes
Thanks, Arne!

But I'm having another problem now. When I transform my data into a
pdata.frame form and try to run a tobit model with random effects I get an
error. Below I provide the head of my data, the code I used and the error
message. Any help?

> head(pdata)
                                   X   X.1 year mun.cod uf.cod uf.name
mun.name       uf.mun
ACACRELANDIA-2003  ACACRELANDIA-2003     1 2003 1200013     12      AC
ACRELANDIA  ACACRELANDIA
ACACRELANDIA-2004    CEIBIAPINA-2003  5561 2004 1200013     12      AC
ACRELANDIA  ACACRELANDIA
ACACRELANDIA-2005     MGARACUAI-2007 11121 2005 1200013     12      AC
ACRELANDIA  ACACRELANDIA
ACACRELANDIA-2006 MGSAO GERALDO-2006 16681 2006 1200013     12      AC
ACRELANDIA  ACACRELANDIA
ACACRELANDIA-2007 PBSERTAOZINHO-2005 22241 2007 1200013     12      AC
ACRELANDIA  ACACRELANDIA
ACACRELANDIA-2008        PRPIEN-2008 27801 2008 1200013     12      AC
ACRELANDIA  ACACRELANDIA
                  transfers populat agric.pib indust.pib servic.pib  apu.pib
 tax.pib       pib
ACACRELANDIA-2003       920    8695  31744.92   5292.439   26942.19 15955.14
4527.178  68506.73
ACACRELANDIA-2004         0   10668  48947.97   9353.529   33134.56 19922.54
5825.801  97261.86
ACACRELANDIA-2005     49287   11451  56291.90  11214.051   39326.32 23340.46
7661.003 114493.30
ACACRELANDIA-2006    385000   11786  55290.26  11976.119   46405.99 27401.64
7730.059 121402.40
ACACRELANDIA-2007   1599730   11520  68478.03  15526.679   54555.67 31382.93
7550.055 146110.40
ACACRELANDIA-2008   1295000   11987  61736.50  12812.245   63688.96 37518.87
7824.615 146062.30
                  bol.fam      ifdm mayor.party mayor.num
  mayor.name mayor.vot
ACACRELANDIA-2003       0 0.4225000        PSDB        45 SEBASTI\xeaO
BOCALOM RODRIGUES      2394
ACACRELANDIA-2004     402 0.3943083        PSDB        45 SEBASTI\xeaO
BOCALOM RODRIGUES      2394
ACACRELANDIA-2005     742 0.5401000        PSDB        45 SEBASTI\xcdO
BOCALOM RODRIGUES      2173
ACACRELANDIA-2006     779 0.5138000        PSDB        45 SEBASTI\xcdO
BOCALOM RODRIGUES      2173
ACACRELANDIA-2007     932 0.5109000        PSDB        45 SEBASTI\xcdO
BOCALOM RODRIGUES      2173
ACACRELANDIA-2008     968 0.5593250        PSDB        45 SEBASTI\xcdO
BOCALOM RODRIGUES      2173
                  mayor.vot.per
 mayor.coalt voters.able
ACACRELANDIA-2003        0.5901 PSDB / PT / PC do B / PV / PMN / PSB / PPS /
PDT        6422
ACACRELANDIA-2004        0.5901 PSDB / PT / PC do B / PV / PMN / PSB / PPS /
PDT        6422
ACACRELANDIA-2005        0.4082               PP / PDT / PMDB / PPS / PFL /
PSDB        6422
ACACRELANDIA-2006        0.4082               PP / PDT / PMDB / PPS / PFL /
PSDB        6422
ACACRELANDIA-2007        0.4082               PP / PDT / PMDB / PPS / PFL /
PSDB        7548
ACACRELANDIA-2008        0.4082               PP / PDT / PMDB / PPS / PFL /
PSDB        7548
                  presd.turnout lula.vot lula.vot.per transfers.cap pt.dummy
pres.turnout.cap
ACACRELANDIA-2003          4282     2202      0.51425     0.1058079        0
       0.4924669
ACACRELANDIA-2004          4282     2202      0.51425     0.0000000        0
       0.4013873
ACACRELANDIA-2005          4282     2202      0.51425     4.3041656        0
       0.3739411
ACACRELANDIA-2006          4282     2202      0.51425    32.6658748        0
       0.3633124
ACACRELANDIA-2007          5461     2422      0.44400   138.8654514        0
       0.4740451
ACACRELANDIA-2008          5461     2422      0.44400   108.0337032        0
       0.4555769
                    pib.cap mun.vote mun.vote2 mun.vote3 mayor.party.r
mayor.party.r2 lula.vot.weight
ACACRELANDIA-2003  7.878865        1        PT      0.PT             0
2.Opposition    5.084315e-06
ACACRELANDIA-2004  9.117160        1        PT      0.PT             0
2.Opposition    5.084315e-06
ACACRELANDIA-2005  9.998542        1        PT      0.PT             0
2.Opposition    5.084315e-06
ACACRELANDIA-2006 10.300560        1        PT      0.PT             0
2.Opposition    5.084315e-06
ACACRELANDIA-2007 12.683194        0      PSDB    1.PSDB             0
2.Opposition    5.592285e-06
ACACRELANDIA-2008 12.185059        0      PSDB    1.PSDB             0
2.Opposition    5.592285e-06
                  trans.dummy transfers.cap.lag     year2 district term
pt.pt pt.coa pt.opp psdb.pt
ACACRELANDIA-2003           1                NA 2003-2004   PT/Opo    0
0      0      1       0
ACACRELANDIA-2004           0                NA 2003-2004   PT/Opo    0
0      0      1       0
ACACRELANDIA-2005           1         474.56074 2005-2006   PT/Opo    0
0      0      1       0
ACACRELANDIA-2006           1          22.94781 2005-2006   PT/Opo    0
0      0      1       0
ACACRELANDIA-2007           1           0.00000 2007-2008 PSDB/Opo    1
0      0      0       0
ACACRELANDIA-2008           1           0.00000 2007-2008 PSDB/Opo    1
0      0      0       0
                  psdb.coa psdb.opp time transf.log
ACACRELANDIA-2003        0        0    1  -2.236723
ACACRELANDIA-2004        0        0    2  -6.907755
ACACRELANDIA-2005        0        0    3   1.459816
ACACRELANDIA-2006        0        0    4   3.486362
ACACRELANDIA-2007        0        1    1   4.933513
ACACRELANDIA-2008        0        1    2   4.682453

tob6 <- censReg(transfers.cap ~ pt.pt + psdb.pt + pt.opp + pt.coa + psdb.coa
+ transfers.cap.lag + pib.cap + ifdm + log(populat) + mayor.vot.per +
bol.fam + factor(uf.name) + factor(year), left=0, right=Inf, method="BHHH",
nGHQ=15, data = pdata)

Error in maxNRCompute(fn = logLikAttr, fnOrig = fn, gradOrig = grad,
hessOrig = hess,  :
  NA in the initial gradient

Thanks!

*Felipe Nunes*
CAPES/Fulbright Fellow
PhD Student Political Science - UCLA
Web: felipenunes.bol.ucla.edu



On Sat, Sep 17, 2011 at 10:44 PM, Arne Henningsen <
[hidden email]> wrote:

> On 18 September 2011 04:58, Felipe Nunes <[hidden email]> wrote:
> > Do you guys know how could I increase the time limit for the iterations
> in
> > censReg? I'm receiving the following message whatever method I use:
> > Newton-Raphson maximisation, 150 iterations
> > Return code 4: Iteration limit exceeded.
> > Log-likelihood: -67680.41 on 42 Df
>
> You can use argument "iterlim" to increase the maximum number of
> iterations, see documentation of maxNR().
>
> /Arne
>
> --
> Arne Henningsen
> http://www.arne-henningsen.name
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Tobit Fixed Effects

Arne Henningsen-4
Hi Felipe

On 18 September 2011 09:09, Felipe Nunes <[hidden email]> wrote:

> Thanks, Arne!
> But I'm having another problem now. When I transform my data into a
> pdata.frame form and try to run a tobit model with random effects I get an
> error. Below I provide the head of my data, the code I used and the error
> message. Any help?
>> head(pdata)
>                                    X   X.1 year mun.cod uf.cod uf.name
>  mun.name       uf.mun
> ACACRELANDIA-2003  ACACRELANDIA-2003     1 2003 1200013     12      AC
> ACRELANDIA  ACACRELANDIA
> ACACRELANDIA-2004    CEIBIAPINA-2003  5561 2004 1200013     12      AC
> ACRELANDIA  ACACRELANDIA
> ACACRELANDIA-2005     MGARACUAI-2007 11121 2005 1200013     12      AC
> ACRELANDIA  ACACRELANDIA
> ACACRELANDIA-2006 MGSAO GERALDO-2006 16681 2006 1200013     12      AC
> ACRELANDIA  ACACRELANDIA
> ACACRELANDIA-2007 PBSERTAOZINHO-2005 22241 2007 1200013     12      AC
> ACRELANDIA  ACACRELANDIA
> ACACRELANDIA-2008        PRPIEN-2008 27801 2008 1200013     12      AC
> ACRELANDIA  ACACRELANDIA
>                   transfers populat agric.pib indust.pib servic.pib  apu.pib
>  tax.pib       pib
> ACACRELANDIA-2003       920    8695  31744.92   5292.439   26942.19 15955.14
> 4527.178  68506.73
> ACACRELANDIA-2004         0   10668  48947.97   9353.529   33134.56 19922.54
> 5825.801  97261.86
> ACACRELANDIA-2005     49287   11451  56291.90  11214.051   39326.32 23340.46
> 7661.003 114493.30
> ACACRELANDIA-2006    385000   11786  55290.26  11976.119   46405.99 27401.64
> 7730.059 121402.40
> ACACRELANDIA-2007   1599730   11520  68478.03  15526.679   54555.67 31382.93
> 7550.055 146110.40
> ACACRELANDIA-2008   1295000   11987  61736.50  12812.245   63688.96 37518.87
> 7824.615 146062.30
>                   bol.fam      ifdm mayor.party mayor.num
>   mayor.name mayor.vot
> ACACRELANDIA-2003       0 0.4225000        PSDB        45 SEBASTI\xeaO
> BOCALOM RODRIGUES      2394
> ACACRELANDIA-2004     402 0.3943083        PSDB        45 SEBASTI\xeaO
> BOCALOM RODRIGUES      2394
> ACACRELANDIA-2005     742 0.5401000        PSDB        45 SEBASTI\xcdO
> BOCALOM RODRIGUES      2173
> ACACRELANDIA-2006     779 0.5138000        PSDB        45 SEBASTI\xcdO
> BOCALOM RODRIGUES      2173
> ACACRELANDIA-2007     932 0.5109000        PSDB        45 SEBASTI\xcdO
> BOCALOM RODRIGUES      2173
> ACACRELANDIA-2008     968 0.5593250        PSDB        45 SEBASTI\xcdO
> BOCALOM RODRIGUES      2173
>                   mayor.vot.per
>  mayor.coalt voters.able
> ACACRELANDIA-2003        0.5901 PSDB / PT / PC do B / PV / PMN / PSB / PPS /
> PDT        6422
> ACACRELANDIA-2004        0.5901 PSDB / PT / PC do B / PV / PMN / PSB / PPS /
> PDT        6422
> ACACRELANDIA-2005        0.4082               PP / PDT / PMDB / PPS / PFL /
> PSDB        6422
> ACACRELANDIA-2006        0.4082               PP / PDT / PMDB / PPS / PFL /
> PSDB        6422
> ACACRELANDIA-2007        0.4082               PP / PDT / PMDB / PPS / PFL /
> PSDB        7548
> ACACRELANDIA-2008        0.4082               PP / PDT / PMDB / PPS / PFL /
> PSDB        7548
>                   presd.turnout lula.vot lula.vot.per transfers.cap pt.dummy
> pres.turnout.cap
> ACACRELANDIA-2003          4282     2202      0.51425     0.1058079        0
>        0.4924669
> ACACRELANDIA-2004          4282     2202      0.51425     0.0000000        0
>        0.4013873
> ACACRELANDIA-2005          4282     2202      0.51425     4.3041656        0
>        0.3739411
> ACACRELANDIA-2006          4282     2202      0.51425    32.6658748        0
>        0.3633124
> ACACRELANDIA-2007          5461     2422      0.44400   138.8654514        0
>        0.4740451
> ACACRELANDIA-2008          5461     2422      0.44400   108.0337032        0
>        0.4555769
>                     pib.cap mun.vote mun.vote2 mun.vote3 mayor.party.r
> mayor.party.r2 lula.vot.weight
> ACACRELANDIA-2003  7.878865        1        PT      0.PT             0
> 2.Opposition    5.084315e-06
> ACACRELANDIA-2004  9.117160        1        PT      0.PT             0
> 2.Opposition    5.084315e-06
> ACACRELANDIA-2005  9.998542        1        PT      0.PT             0
> 2.Opposition    5.084315e-06
> ACACRELANDIA-2006 10.300560        1        PT      0.PT             0
> 2.Opposition    5.084315e-06
> ACACRELANDIA-2007 12.683194        0      PSDB    1.PSDB             0
> 2.Opposition    5.592285e-06
> ACACRELANDIA-2008 12.185059        0      PSDB    1.PSDB             0
> 2.Opposition    5.592285e-06
>                   trans.dummy transfers.cap.lag     year2 district term
> pt.pt pt.coa pt.opp psdb.pt
> ACACRELANDIA-2003           1                NA 2003-2004   PT/Opo    0
> 0      0      1       0
> ACACRELANDIA-2004           0                NA 2003-2004   PT/Opo    0
> 0      0      1       0
> ACACRELANDIA-2005           1         474.56074 2005-2006   PT/Opo    0
> 0      0      1       0
> ACACRELANDIA-2006           1          22.94781 2005-2006   PT/Opo    0
> 0      0      1       0
> ACACRELANDIA-2007           1           0.00000 2007-2008 PSDB/Opo    1
> 0      0      0       0
> ACACRELANDIA-2008           1           0.00000 2007-2008 PSDB/Opo    1
> 0      0      0       0
>                   psdb.coa psdb.opp time transf.log
> ACACRELANDIA-2003        0        0    1  -2.236723
> ACACRELANDIA-2004        0        0    2  -6.907755
> ACACRELANDIA-2005        0        0    3   1.459816
> ACACRELANDIA-2006        0        0    4   3.486362
> ACACRELANDIA-2007        0        1    1   4.933513
> ACACRELANDIA-2008        0        1    2   4.682453
> tob6 <- censReg(transfers.cap ~ pt.pt + psdb.pt + pt.opp + pt.coa + psdb.coa
> + transfers.cap.lag + pib.cap + ifdm + log(populat) + mayor.vot.per +
> bol.fam + factor(uf.name) + factor(year), left=0, right=Inf, method="BHHH",
> nGHQ=15, data = pdata)
> Error in maxNRCompute(fn = logLikAttr, fnOrig = fn, gradOrig = grad,
> hessOrig = hess,  :
>   NA in the initial gradient

Please try to use the latest development version of the censReg
package, i.e. version 0.5-7 which is available at R-Forge [1] (see
[2], [3], and [4]).

[1] https://r-forge.r-project.org/R/?group_id=256
[2] http://tolstoy.newcastle.edu.au/R/e15/help/11/09/7037.html
[3] http://tolstoy.newcastle.edu.au/R/e15/help/11/09/7288.html
[4] http://tolstoy.newcastle.edu.au/R/e15/help/11/09/7307.html

/Arne

--
Arne Henningsen
http://www.arne-henningsen.name

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Tobit Fixed Effects

felipnunes
Thanks!

*Felipe Nunes*
CAPES/Fulbright Fellow
PhD Student Political Science - UCLA
Web: felipenunes.bol.ucla.edu



On Sun, Sep 18, 2011 at 2:19 AM, Arne Henningsen <
[hidden email]> wrote:

> Hi Felipe
>
> On 18 September 2011 09:09, Felipe Nunes <[hidden email]> wrote:
> > Thanks, Arne!
> > But I'm having another problem now. When I transform my data into a
> > pdata.frame form and try to run a tobit model with random effects I get
> an
> > error. Below I provide the head of my data, the code I used and the error
> > message. Any help?
> >> head(pdata)
> >                                    X   X.1 year mun.cod uf.cod uf.name
> >  mun.name       uf.mun
> > ACACRELANDIA-2003  ACACRELANDIA-2003     1 2003 1200013     12      AC
> > ACRELANDIA  ACACRELANDIA
> > ACACRELANDIA-2004    CEIBIAPINA-2003  5561 2004 1200013     12      AC
> > ACRELANDIA  ACACRELANDIA
> > ACACRELANDIA-2005     MGARACUAI-2007 11121 2005 1200013     12      AC
> > ACRELANDIA  ACACRELANDIA
> > ACACRELANDIA-2006 MGSAO GERALDO-2006 16681 2006 1200013     12      AC
> > ACRELANDIA  ACACRELANDIA
> > ACACRELANDIA-2007 PBSERTAOZINHO-2005 22241 2007 1200013     12      AC
> > ACRELANDIA  ACACRELANDIA
> > ACACRELANDIA-2008        PRPIEN-2008 27801 2008 1200013     12      AC
> > ACRELANDIA  ACACRELANDIA
> >                   transfers populat agric.pib indust.pib servic.pib
>  apu.pib
> >  tax.pib       pib
> > ACACRELANDIA-2003       920    8695  31744.92   5292.439   26942.19
> 15955.14
> > 4527.178  68506.73
> > ACACRELANDIA-2004         0   10668  48947.97   9353.529   33134.56
> 19922.54
> > 5825.801  97261.86
> > ACACRELANDIA-2005     49287   11451  56291.90  11214.051   39326.32
> 23340.46
> > 7661.003 114493.30
> > ACACRELANDIA-2006    385000   11786  55290.26  11976.119   46405.99
> 27401.64
> > 7730.059 121402.40
> > ACACRELANDIA-2007   1599730   11520  68478.03  15526.679   54555.67
> 31382.93
> > 7550.055 146110.40
> > ACACRELANDIA-2008   1295000   11987  61736.50  12812.245   63688.96
> 37518.87
> > 7824.615 146062.30
> >                   bol.fam      ifdm mayor.party mayor.num
> >   mayor.name mayor.vot
> > ACACRELANDIA-2003       0 0.4225000        PSDB        45 SEBASTI\xeaO
> > BOCALOM RODRIGUES      2394
> > ACACRELANDIA-2004     402 0.3943083        PSDB        45 SEBASTI\xeaO
> > BOCALOM RODRIGUES      2394
> > ACACRELANDIA-2005     742 0.5401000        PSDB        45 SEBASTI\xcdO
> > BOCALOM RODRIGUES      2173
> > ACACRELANDIA-2006     779 0.5138000        PSDB        45 SEBASTI\xcdO
> > BOCALOM RODRIGUES      2173
> > ACACRELANDIA-2007     932 0.5109000        PSDB        45 SEBASTI\xcdO
> > BOCALOM RODRIGUES      2173
> > ACACRELANDIA-2008     968 0.5593250        PSDB        45 SEBASTI\xcdO
> > BOCALOM RODRIGUES      2173
> >                   mayor.vot.per
> >  mayor.coalt voters.able
> > ACACRELANDIA-2003        0.5901 PSDB / PT / PC do B / PV / PMN / PSB /
> PPS /
> > PDT        6422
> > ACACRELANDIA-2004        0.5901 PSDB / PT / PC do B / PV / PMN / PSB /
> PPS /
> > PDT        6422
> > ACACRELANDIA-2005        0.4082               PP / PDT / PMDB / PPS / PFL
> /
> > PSDB        6422
> > ACACRELANDIA-2006        0.4082               PP / PDT / PMDB / PPS / PFL
> /
> > PSDB        6422
> > ACACRELANDIA-2007        0.4082               PP / PDT / PMDB / PPS / PFL
> /
> > PSDB        7548
> > ACACRELANDIA-2008        0.4082               PP / PDT / PMDB / PPS / PFL
> /
> > PSDB        7548
> >                   presd.turnout lula.vot lula.vot.per transfers.cap
> pt.dummy
> > pres.turnout.cap
> > ACACRELANDIA-2003          4282     2202      0.51425     0.1058079
>  0
> >        0.4924669
> > ACACRELANDIA-2004          4282     2202      0.51425     0.0000000
>  0
> >        0.4013873
> > ACACRELANDIA-2005          4282     2202      0.51425     4.3041656
>  0
> >        0.3739411
> > ACACRELANDIA-2006          4282     2202      0.51425    32.6658748
>  0
> >        0.3633124
> > ACACRELANDIA-2007          5461     2422      0.44400   138.8654514
>  0
> >        0.4740451
> > ACACRELANDIA-2008          5461     2422      0.44400   108.0337032
>  0
> >        0.4555769
> >                     pib.cap mun.vote mun.vote2 mun.vote3 mayor.party.r
> > mayor.party.r2 lula.vot.weight
> > ACACRELANDIA-2003  7.878865        1        PT      0.PT             0
> > 2.Opposition    5.084315e-06
> > ACACRELANDIA-2004  9.117160        1        PT      0.PT             0
> > 2.Opposition    5.084315e-06
> > ACACRELANDIA-2005  9.998542        1        PT      0.PT             0
> > 2.Opposition    5.084315e-06
> > ACACRELANDIA-2006 10.300560        1        PT      0.PT             0
> > 2.Opposition    5.084315e-06
> > ACACRELANDIA-2007 12.683194        0      PSDB    1.PSDB             0
> > 2.Opposition    5.592285e-06
> > ACACRELANDIA-2008 12.185059        0      PSDB    1.PSDB             0
> > 2.Opposition    5.592285e-06
> >                   trans.dummy transfers.cap.lag     year2 district term
> > pt.pt pt.coa pt.opp psdb.pt
> > ACACRELANDIA-2003           1                NA 2003-2004   PT/Opo    0
> > 0      0      1       0
> > ACACRELANDIA-2004           0                NA 2003-2004   PT/Opo    0
> > 0      0      1       0
> > ACACRELANDIA-2005           1         474.56074 2005-2006   PT/Opo    0
> > 0      0      1       0
> > ACACRELANDIA-2006           1          22.94781 2005-2006   PT/Opo    0
> > 0      0      1       0
> > ACACRELANDIA-2007           1           0.00000 2007-2008 PSDB/Opo    1
> > 0      0      0       0
> > ACACRELANDIA-2008           1           0.00000 2007-2008 PSDB/Opo    1
> > 0      0      0       0
> >                   psdb.coa psdb.opp time transf.log
> > ACACRELANDIA-2003        0        0    1  -2.236723
> > ACACRELANDIA-2004        0        0    2  -6.907755
> > ACACRELANDIA-2005        0        0    3   1.459816
> > ACACRELANDIA-2006        0        0    4   3.486362
> > ACACRELANDIA-2007        0        1    1   4.933513
> > ACACRELANDIA-2008        0        1    2   4.682453
> > tob6 <- censReg(transfers.cap ~ pt.pt + psdb.pt + pt.opp + pt.coa +
> psdb.coa
> > + transfers.cap.lag + pib.cap + ifdm + log(populat) + mayor.vot.per +
> > bol.fam + factor(uf.name) + factor(year), left=0, right=Inf,
> method="BHHH",
> > nGHQ=15, data = pdata)
> > Error in maxNRCompute(fn = logLikAttr, fnOrig = fn, gradOrig = grad,
> > hessOrig = hess,  :
> >   NA in the initial gradient
>
> Please try to use the latest development version of the censReg
> package, i.e. version 0.5-7 which is available at R-Forge [1] (see
> [2], [3], and [4]).
>
> [1] https://r-forge.r-project.org/R/?group_id=256
> [2] http://tolstoy.newcastle.edu.au/R/e15/help/11/09/7037.html
> [3] http://tolstoy.newcastle.edu.au/R/e15/help/11/09/7288.html
> [4] http://tolstoy.newcastle.edu.au/R/e15/help/11/09/7307.html
>
> /Arne
>
> --
> Arne Henningsen
> http://www.arne-henningsen.name
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Tobit Fixed Effects

felipnunes
Hi Arne,

my problem persists. I am still using censReg [version - 0.5-7] to run a
random effects model in my data (>50,000 cases), but I always get the
message.

tob7 <- censReg(transfers.cap ~ pt.pt + psdb.pt + pt.opp + pt.coa + psdb.coa
+ pib.cap + transfers.cap.lag + pib.cap + ifdm + log(populat) +
mayor.vot.per + log(bol.fam.per+0.01) + factor(uf.name) + factor(year) - 1,
left=0, right=Inf, method="BHHH", nGHQ=8, iterlim=10000, data = pdata2)

Error in maxNRCompute(fn = logLikAttr, fnOrig = fn, gradOrig = grad,
hessOrig = hess,  :
  NA in the initial gradient

If I sent you my data set, could you try to help me? I have been struggling
with that for two months now.

I hope you can help!

Thanks,

*Felipe Nunes*
CAPES/Fulbright Fellow
PhD Student Political Science - UCLA
Web: felipenunes.bol.ucla.edu



On Sun, Sep 18, 2011 at 9:42 AM, Felipe Nunes <[hidden email]> wrote:

> Thanks!
>
> *Felipe Nunes*
> CAPES/Fulbright Fellow
> PhD Student Political Science - UCLA
> Web: felipenunes.bol.ucla.edu
>
>
>
> On Sun, Sep 18, 2011 at 2:19 AM, Arne Henningsen <
> [hidden email]> wrote:
>
>> Hi Felipe
>>
>> On 18 September 2011 09:09, Felipe Nunes <[hidden email]> wrote:
>> > Thanks, Arne!
>> > But I'm having another problem now. When I transform my data into a
>> > pdata.frame form and try to run a tobit model with random effects I get
>> an
>> > error. Below I provide the head of my data, the code I used and the
>> error
>> > message. Any help?
>> >> head(pdata)
>> >                                    X   X.1 year mun.cod uf.cod uf.name
>> >  mun.name       uf.mun
>> > ACACRELANDIA-2003  ACACRELANDIA-2003     1 2003 1200013     12      AC
>> > ACRELANDIA  ACACRELANDIA
>> > ACACRELANDIA-2004    CEIBIAPINA-2003  5561 2004 1200013     12      AC
>> > ACRELANDIA  ACACRELANDIA
>> > ACACRELANDIA-2005     MGARACUAI-2007 11121 2005 1200013     12      AC
>> > ACRELANDIA  ACACRELANDIA
>> > ACACRELANDIA-2006 MGSAO GERALDO-2006 16681 2006 1200013     12      AC
>> > ACRELANDIA  ACACRELANDIA
>> > ACACRELANDIA-2007 PBSERTAOZINHO-2005 22241 2007 1200013     12      AC
>> > ACRELANDIA  ACACRELANDIA
>> > ACACRELANDIA-2008        PRPIEN-2008 27801 2008 1200013     12      AC
>> > ACRELANDIA  ACACRELANDIA
>> >                   transfers populat agric.pib indust.pib servic.pib
>>  apu.pib
>> >  tax.pib       pib
>> > ACACRELANDIA-2003       920    8695  31744.92   5292.439   26942.19
>> 15955.14
>> > 4527.178  68506.73
>> > ACACRELANDIA-2004         0   10668  48947.97   9353.529   33134.56
>> 19922.54
>> > 5825.801  97261.86
>> > ACACRELANDIA-2005     49287   11451  56291.90  11214.051   39326.32
>> 23340.46
>> > 7661.003 114493.30
>> > ACACRELANDIA-2006    385000   11786  55290.26  11976.119   46405.99
>> 27401.64
>> > 7730.059 121402.40
>> > ACACRELANDIA-2007   1599730   11520  68478.03  15526.679   54555.67
>> 31382.93
>> > 7550.055 146110.40
>> > ACACRELANDIA-2008   1295000   11987  61736.50  12812.245   63688.96
>> 37518.87
>> > 7824.615 146062.30
>> >                   bol.fam      ifdm mayor.party mayor.num
>> >   mayor.name mayor.vot
>> > ACACRELANDIA-2003       0 0.4225000        PSDB        45 SEBASTI\xeaO
>> > BOCALOM RODRIGUES      2394
>> > ACACRELANDIA-2004     402 0.3943083        PSDB        45 SEBASTI\xeaO
>> > BOCALOM RODRIGUES      2394
>> > ACACRELANDIA-2005     742 0.5401000        PSDB        45 SEBASTI\xcdO
>> > BOCALOM RODRIGUES      2173
>> > ACACRELANDIA-2006     779 0.5138000        PSDB        45 SEBASTI\xcdO
>> > BOCALOM RODRIGUES      2173
>> > ACACRELANDIA-2007     932 0.5109000        PSDB        45 SEBASTI\xcdO
>> > BOCALOM RODRIGUES      2173
>> > ACACRELANDIA-2008     968 0.5593250        PSDB        45 SEBASTI\xcdO
>> > BOCALOM RODRIGUES      2173
>> >                   mayor.vot.per
>> >  mayor.coalt voters.able
>> > ACACRELANDIA-2003        0.5901 PSDB / PT / PC do B / PV / PMN / PSB /
>> PPS /
>> > PDT        6422
>> > ACACRELANDIA-2004        0.5901 PSDB / PT / PC do B / PV / PMN / PSB /
>> PPS /
>> > PDT        6422
>> > ACACRELANDIA-2005        0.4082               PP / PDT / PMDB / PPS /
>> PFL /
>> > PSDB        6422
>> > ACACRELANDIA-2006        0.4082               PP / PDT / PMDB / PPS /
>> PFL /
>> > PSDB        6422
>> > ACACRELANDIA-2007        0.4082               PP / PDT / PMDB / PPS /
>> PFL /
>> > PSDB        7548
>> > ACACRELANDIA-2008        0.4082               PP / PDT / PMDB / PPS /
>> PFL /
>> > PSDB        7548
>> >                   presd.turnout lula.vot lula.vot.per transfers.cap
>> pt.dummy
>> > pres.turnout.cap
>> > ACACRELANDIA-2003          4282     2202      0.51425     0.1058079
>>    0
>> >        0.4924669
>> > ACACRELANDIA-2004          4282     2202      0.51425     0.0000000
>>    0
>> >        0.4013873
>> > ACACRELANDIA-2005          4282     2202      0.51425     4.3041656
>>    0
>> >        0.3739411
>> > ACACRELANDIA-2006          4282     2202      0.51425    32.6658748
>>    0
>> >        0.3633124
>> > ACACRELANDIA-2007          5461     2422      0.44400   138.8654514
>>    0
>> >        0.4740451
>> > ACACRELANDIA-2008          5461     2422      0.44400   108.0337032
>>    0
>> >        0.4555769
>> >                     pib.cap mun.vote mun.vote2 mun.vote3 mayor.party.r
>> > mayor.party.r2 lula.vot.weight
>> > ACACRELANDIA-2003  7.878865        1        PT      0.PT             0
>> > 2.Opposition    5.084315e-06
>> > ACACRELANDIA-2004  9.117160        1        PT      0.PT             0
>> > 2.Opposition    5.084315e-06
>> > ACACRELANDIA-2005  9.998542        1        PT      0.PT             0
>> > 2.Opposition    5.084315e-06
>> > ACACRELANDIA-2006 10.300560        1        PT      0.PT             0
>> > 2.Opposition    5.084315e-06
>> > ACACRELANDIA-2007 12.683194        0      PSDB    1.PSDB             0
>> > 2.Opposition    5.592285e-06
>> > ACACRELANDIA-2008 12.185059        0      PSDB    1.PSDB             0
>> > 2.Opposition    5.592285e-06
>> >                   trans.dummy transfers.cap.lag     year2 district term
>> > pt.pt pt.coa pt.opp psdb.pt
>> > ACACRELANDIA-2003           1                NA 2003-2004   PT/Opo    0
>> > 0      0      1       0
>> > ACACRELANDIA-2004           0                NA 2003-2004   PT/Opo    0
>> > 0      0      1       0
>> > ACACRELANDIA-2005           1         474.56074 2005-2006   PT/Opo    0
>> > 0      0      1       0
>> > ACACRELANDIA-2006           1          22.94781 2005-2006   PT/Opo    0
>> > 0      0      1       0
>> > ACACRELANDIA-2007           1           0.00000 2007-2008 PSDB/Opo    1
>> > 0      0      0       0
>> > ACACRELANDIA-2008           1           0.00000 2007-2008 PSDB/Opo    1
>> > 0      0      0       0
>> >                   psdb.coa psdb.opp time transf.log
>> > ACACRELANDIA-2003        0        0    1  -2.236723
>> > ACACRELANDIA-2004        0        0    2  -6.907755
>> > ACACRELANDIA-2005        0        0    3   1.459816
>> > ACACRELANDIA-2006        0        0    4   3.486362
>> > ACACRELANDIA-2007        0        1    1   4.933513
>> > ACACRELANDIA-2008        0        1    2   4.682453
>> > tob6 <- censReg(transfers.cap ~ pt.pt + psdb.pt + pt.opp + pt.coa +
>> psdb.coa
>> > + transfers.cap.lag + pib.cap + ifdm + log(populat) + mayor.vot.per +
>> > bol.fam + factor(uf.name) + factor(year), left=0, right=Inf,
>> method="BHHH",
>> > nGHQ=15, data = pdata)
>> > Error in maxNRCompute(fn = logLikAttr, fnOrig = fn, gradOrig = grad,
>> > hessOrig = hess,  :
>> >   NA in the initial gradient
>>
>> Please try to use the latest development version of the censReg
>> package, i.e. version 0.5-7 which is available at R-Forge [1] (see
>> [2], [3], and [4]).
>>
>> [1] https://r-forge.r-project.org/R/?group_id=256
>> [2] http://tolstoy.newcastle.edu.au/R/e15/help/11/09/7037.html
>> [3] http://tolstoy.newcastle.edu.au/R/e15/help/11/09/7288.html
>> [4] http://tolstoy.newcastle.edu.au/R/e15/help/11/09/7307.html
>>
>> /Arne
>>
>> --
>> Arne Henningsen
>> http://www.arne-henningsen.name
>>
>
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Tobit Fixed Effects

Joshua Wiley-2
Can you fit a simpler model successfully?  I would play around with
all your RHS variables to try to see if there is any particular one
that causes the estimation problems.  Two ways I would do this would
be to drop them one at a time, or to start from an intercept only
model (i.e., transfers.cap ~ 1) and if that runs successfully, build
up one at a time until I started to get errors.  I would also, to the
extent possible, look at the individual and joint distributions of the
variables.  Missing cases are probably dropped before starting the
estimation, so looking at the joint distributions on na.omit(pdata2)
may be more fruitful than on pdata2 itself.

Cheers,

Josh

On Sat, Sep 24, 2011 at 3:16 PM, Felipe Nunes <[hidden email]> wrote:

> Hi Arne,
>
> my problem persists. I am still using censReg [version - 0.5-7] to run a
> random effects model in my data (>50,000 cases), but I always get the
> message.
>
> tob7 <- censReg(transfers.cap ~ pt.pt + psdb.pt + pt.opp + pt.coa + psdb.coa
> + pib.cap + transfers.cap.lag + pib.cap + ifdm + log(populat) +
> mayor.vot.per + log(bol.fam.per+0.01) + factor(uf.name) + factor(year) - 1,
> left=0, right=Inf, method="BHHH", nGHQ=8, iterlim=10000, data = pdata2)
>
> Error in maxNRCompute(fn = logLikAttr, fnOrig = fn, gradOrig = grad,
> hessOrig = hess,  :
>  NA in the initial gradient
>
> If I sent you my data set, could you try to help me? I have been struggling
> with that for two months now.
>
> I hope you can help!
>
> Thanks,
>
> *Felipe Nunes*
> CAPES/Fulbright Fellow
> PhD Student Political Science - UCLA
> Web: felipenunes.bol.ucla.edu
>
>
>
> On Sun, Sep 18, 2011 at 9:42 AM, Felipe Nunes <[hidden email]> wrote:
>
>> Thanks!
>>
>> *Felipe Nunes*
>> CAPES/Fulbright Fellow
>> PhD Student Political Science - UCLA
>> Web: felipenunes.bol.ucla.edu
>>
>>
>>
>> On Sun, Sep 18, 2011 at 2:19 AM, Arne Henningsen <
>> [hidden email]> wrote:
>>
>>> Hi Felipe
>>>
>>> On 18 September 2011 09:09, Felipe Nunes <[hidden email]> wrote:
>>> > Thanks, Arne!
>>> > But I'm having another problem now. When I transform my data into a
>>> > pdata.frame form and try to run a tobit model with random effects I get
>>> an
>>> > error. Below I provide the head of my data, the code I used and the
>>> error
>>> > message. Any help?
>>> >> head(pdata)
>>> >                                    X   X.1 year mun.cod uf.cod uf.name
>>> >  mun.name       uf.mun
>>> > ACACRELANDIA-2003  ACACRELANDIA-2003     1 2003 1200013     12      AC
>>> > ACRELANDIA  ACACRELANDIA
>>> > ACACRELANDIA-2004    CEIBIAPINA-2003  5561 2004 1200013     12      AC
>>> > ACRELANDIA  ACACRELANDIA
>>> > ACACRELANDIA-2005     MGARACUAI-2007 11121 2005 1200013     12      AC
>>> > ACRELANDIA  ACACRELANDIA
>>> > ACACRELANDIA-2006 MGSAO GERALDO-2006 16681 2006 1200013     12      AC
>>> > ACRELANDIA  ACACRELANDIA
>>> > ACACRELANDIA-2007 PBSERTAOZINHO-2005 22241 2007 1200013     12      AC
>>> > ACRELANDIA  ACACRELANDIA
>>> > ACACRELANDIA-2008        PRPIEN-2008 27801 2008 1200013     12      AC
>>> > ACRELANDIA  ACACRELANDIA
>>> >                   transfers populat agric.pib indust.pib servic.pib
>>>  apu.pib
>>> >  tax.pib       pib
>>> > ACACRELANDIA-2003       920    8695  31744.92   5292.439   26942.19
>>> 15955.14
>>> > 4527.178  68506.73
>>> > ACACRELANDIA-2004         0   10668  48947.97   9353.529   33134.56
>>> 19922.54
>>> > 5825.801  97261.86
>>> > ACACRELANDIA-2005     49287   11451  56291.90  11214.051   39326.32
>>> 23340.46
>>> > 7661.003 114493.30
>>> > ACACRELANDIA-2006    385000   11786  55290.26  11976.119   46405.99
>>> 27401.64
>>> > 7730.059 121402.40
>>> > ACACRELANDIA-2007   1599730   11520  68478.03  15526.679   54555.67
>>> 31382.93
>>> > 7550.055 146110.40
>>> > ACACRELANDIA-2008   1295000   11987  61736.50  12812.245   63688.96
>>> 37518.87
>>> > 7824.615 146062.30
>>> >                   bol.fam      ifdm mayor.party mayor.num
>>> >   mayor.name mayor.vot
>>> > ACACRELANDIA-2003       0 0.4225000        PSDB        45 SEBASTI\xeaO
>>> > BOCALOM RODRIGUES      2394
>>> > ACACRELANDIA-2004     402 0.3943083        PSDB        45 SEBASTI\xeaO
>>> > BOCALOM RODRIGUES      2394
>>> > ACACRELANDIA-2005     742 0.5401000        PSDB        45 SEBASTI\xcdO
>>> > BOCALOM RODRIGUES      2173
>>> > ACACRELANDIA-2006     779 0.5138000        PSDB        45 SEBASTI\xcdO
>>> > BOCALOM RODRIGUES      2173
>>> > ACACRELANDIA-2007     932 0.5109000        PSDB        45 SEBASTI\xcdO
>>> > BOCALOM RODRIGUES      2173
>>> > ACACRELANDIA-2008     968 0.5593250        PSDB        45 SEBASTI\xcdO
>>> > BOCALOM RODRIGUES      2173
>>> >                   mayor.vot.per
>>> >  mayor.coalt voters.able
>>> > ACACRELANDIA-2003        0.5901 PSDB / PT / PC do B / PV / PMN / PSB /
>>> PPS /
>>> > PDT        6422
>>> > ACACRELANDIA-2004        0.5901 PSDB / PT / PC do B / PV / PMN / PSB /
>>> PPS /
>>> > PDT        6422
>>> > ACACRELANDIA-2005        0.4082               PP / PDT / PMDB / PPS /
>>> PFL /
>>> > PSDB        6422
>>> > ACACRELANDIA-2006        0.4082               PP / PDT / PMDB / PPS /
>>> PFL /
>>> > PSDB        6422
>>> > ACACRELANDIA-2007        0.4082               PP / PDT / PMDB / PPS /
>>> PFL /
>>> > PSDB        7548
>>> > ACACRELANDIA-2008        0.4082               PP / PDT / PMDB / PPS /
>>> PFL /
>>> > PSDB        7548
>>> >                   presd.turnout lula.vot lula.vot.per transfers.cap
>>> pt.dummy
>>> > pres.turnout.cap
>>> > ACACRELANDIA-2003          4282     2202      0.51425     0.1058079
>>>    0
>>> >        0.4924669
>>> > ACACRELANDIA-2004          4282     2202      0.51425     0.0000000
>>>    0
>>> >        0.4013873
>>> > ACACRELANDIA-2005          4282     2202      0.51425     4.3041656
>>>    0
>>> >        0.3739411
>>> > ACACRELANDIA-2006          4282     2202      0.51425    32.6658748
>>>    0
>>> >        0.3633124
>>> > ACACRELANDIA-2007          5461     2422      0.44400   138.8654514
>>>    0
>>> >        0.4740451
>>> > ACACRELANDIA-2008          5461     2422      0.44400   108.0337032
>>>    0
>>> >        0.4555769
>>> >                     pib.cap mun.vote mun.vote2 mun.vote3 mayor.party.r
>>> > mayor.party.r2 lula.vot.weight
>>> > ACACRELANDIA-2003  7.878865        1        PT      0.PT             0
>>> > 2.Opposition    5.084315e-06
>>> > ACACRELANDIA-2004  9.117160        1        PT      0.PT             0
>>> > 2.Opposition    5.084315e-06
>>> > ACACRELANDIA-2005  9.998542        1        PT      0.PT             0
>>> > 2.Opposition    5.084315e-06
>>> > ACACRELANDIA-2006 10.300560        1        PT      0.PT             0
>>> > 2.Opposition    5.084315e-06
>>> > ACACRELANDIA-2007 12.683194        0      PSDB    1.PSDB             0
>>> > 2.Opposition    5.592285e-06
>>> > ACACRELANDIA-2008 12.185059        0      PSDB    1.PSDB             0
>>> > 2.Opposition    5.592285e-06
>>> >                   trans.dummy transfers.cap.lag     year2 district term
>>> > pt.pt pt.coa pt.opp psdb.pt
>>> > ACACRELANDIA-2003           1                NA 2003-2004   PT/Opo    0
>>> > 0      0      1       0
>>> > ACACRELANDIA-2004           0                NA 2003-2004   PT/Opo    0
>>> > 0      0      1       0
>>> > ACACRELANDIA-2005           1         474.56074 2005-2006   PT/Opo    0
>>> > 0      0      1       0
>>> > ACACRELANDIA-2006           1          22.94781 2005-2006   PT/Opo    0
>>> > 0      0      1       0
>>> > ACACRELANDIA-2007           1           0.00000 2007-2008 PSDB/Opo    1
>>> > 0      0      0       0
>>> > ACACRELANDIA-2008           1           0.00000 2007-2008 PSDB/Opo    1
>>> > 0      0      0       0
>>> >                   psdb.coa psdb.opp time transf.log
>>> > ACACRELANDIA-2003        0        0    1  -2.236723
>>> > ACACRELANDIA-2004        0        0    2  -6.907755
>>> > ACACRELANDIA-2005        0        0    3   1.459816
>>> > ACACRELANDIA-2006        0        0    4   3.486362
>>> > ACACRELANDIA-2007        0        1    1   4.933513
>>> > ACACRELANDIA-2008        0        1    2   4.682453
>>> > tob6 <- censReg(transfers.cap ~ pt.pt + psdb.pt + pt.opp + pt.coa +
>>> psdb.coa
>>> > + transfers.cap.lag + pib.cap + ifdm + log(populat) + mayor.vot.per +
>>> > bol.fam + factor(uf.name) + factor(year), left=0, right=Inf,
>>> method="BHHH",
>>> > nGHQ=15, data = pdata)
>>> > Error in maxNRCompute(fn = logLikAttr, fnOrig = fn, gradOrig = grad,
>>> > hessOrig = hess,  :
>>> >   NA in the initial gradient
>>>
>>> Please try to use the latest development version of the censReg
>>> package, i.e. version 0.5-7 which is available at R-Forge [1] (see
>>> [2], [3], and [4]).
>>>
>>> [1] https://r-forge.r-project.org/R/?group_id=256
>>> [2] http://tolstoy.newcastle.edu.au/R/e15/help/11/09/7037.html
>>> [3] http://tolstoy.newcastle.edu.au/R/e15/help/11/09/7288.html
>>> [4] http://tolstoy.newcastle.edu.au/R/e15/help/11/09/7307.html
>>>
>>> /Arne
>>>
>>> --
>>> Arne Henningsen
>>> http://www.arne-henningsen.name
>>>
>>
>>
>
>        [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



--
Joshua Wiley
Ph.D. Student, Health Psychology
Programmer Analyst II, ATS Statistical Consulting Group
University of California, Los Angeles
https://joshuawiley.com/

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Tobit Fixed Effects

felipnunes
That was a great idea, Josh!

I found a problem even when I tried a simple model. Do you have any idea
why?

tob6 <- censReg(transfers.cap ~ 1, left=0, right=Inf, method="BHHH",
nGHQ=20, iterlim=10000, data = pdata2)

Error in uniqval[as.character(effect), , drop = F] :
  incorrect number of dimensions

thanks!

*Felipe Nunes*
CAPES/Fulbright Fellow
PhD Student Political Science - UCLA
Web: felipenunes.bol.ucla.edu



On Sat, Sep 24, 2011 at 3:38 PM, Joshua Wiley <[hidden email]>wrote:

> Can you fit a simpler model successfully?  I would play around with
> all your RHS variables to try to see if there is any particular one
> that causes the estimation problems.  Two ways I would do this would
> be to drop them one at a time, or to start from an intercept only
> model (i.e., transfers.cap ~ 1) and if that runs successfully, build
> up one at a time until I started to get errors.  I would also, to the
> extent possible, look at the individual and joint distributions of the
> variables.  Missing cases are probably dropped before starting the
> estimation, so looking at the joint distributions on na.omit(pdata2)
> may be more fruitful than on pdata2 itself.
>
> Cheers,
>
> Josh
>
> On Sat, Sep 24, 2011 at 3:16 PM, Felipe Nunes <[hidden email]>
> wrote:
> > Hi Arne,
> >
> > my problem persists. I am still using censReg [version - 0.5-7] to run a
> > random effects model in my data (>50,000 cases), but I always get the
> > message.
> >
> > tob7 <- censReg(transfers.cap ~ pt.pt + psdb.pt + pt.opp + pt.coa +
> psdb.coa
> > + pib.cap + transfers.cap.lag + pib.cap + ifdm + log(populat) +
> > mayor.vot.per + log(bol.fam.per+0.01) + factor(uf.name) + factor(year) -
> 1,
> > left=0, right=Inf, method="BHHH", nGHQ=8, iterlim=10000, data = pdata2)
> >
> > Error in maxNRCompute(fn = logLikAttr, fnOrig = fn, gradOrig = grad,
> > hessOrig = hess,  :
> >  NA in the initial gradient
> >
> > If I sent you my data set, could you try to help me? I have been
> struggling
> > with that for two months now.
> >
> > I hope you can help!
> >
> > Thanks,
> >
> > *Felipe Nunes*
> > CAPES/Fulbright Fellow
> > PhD Student Political Science - UCLA
> > Web: felipenunes.bol.ucla.edu
> >
> >
> >
> > On Sun, Sep 18, 2011 at 9:42 AM, Felipe Nunes <[hidden email]>
> wrote:
> >
> >> Thanks!
> >>
> >> *Felipe Nunes*
> >> CAPES/Fulbright Fellow
> >> PhD Student Political Science - UCLA
> >> Web: felipenunes.bol.ucla.edu
> >>
> >>
> >>
> >> On Sun, Sep 18, 2011 at 2:19 AM, Arne Henningsen <
> >> [hidden email]> wrote:
> >>
> >>> Hi Felipe
> >>>
> >>> On 18 September 2011 09:09, Felipe Nunes <[hidden email]> wrote:
> >>> > Thanks, Arne!
> >>> > But I'm having another problem now. When I transform my data into a
> >>> > pdata.frame form and try to run a tobit model with random effects I
> get
> >>> an
> >>> > error. Below I provide the head of my data, the code I used and the
> >>> error
> >>> > message. Any help?
> >>> >> head(pdata)
> >>> >                                    X   X.1 year mun.cod uf.cod
> uf.name
> >>> >  mun.name       uf.mun
> >>> > ACACRELANDIA-2003  ACACRELANDIA-2003     1 2003 1200013     12
>  AC
> >>> > ACRELANDIA  ACACRELANDIA
> >>> > ACACRELANDIA-2004    CEIBIAPINA-2003  5561 2004 1200013     12
>  AC
> >>> > ACRELANDIA  ACACRELANDIA
> >>> > ACACRELANDIA-2005     MGARACUAI-2007 11121 2005 1200013     12
>  AC
> >>> > ACRELANDIA  ACACRELANDIA
> >>> > ACACRELANDIA-2006 MGSAO GERALDO-2006 16681 2006 1200013     12
>  AC
> >>> > ACRELANDIA  ACACRELANDIA
> >>> > ACACRELANDIA-2007 PBSERTAOZINHO-2005 22241 2007 1200013     12
>  AC
> >>> > ACRELANDIA  ACACRELANDIA
> >>> > ACACRELANDIA-2008        PRPIEN-2008 27801 2008 1200013     12
>  AC
> >>> > ACRELANDIA  ACACRELANDIA
> >>> >                   transfers populat agric.pib indust.pib servic.pib
> >>>  apu.pib
> >>> >  tax.pib       pib
> >>> > ACACRELANDIA-2003       920    8695  31744.92   5292.439   26942.19
> >>> 15955.14
> >>> > 4527.178  68506.73
> >>> > ACACRELANDIA-2004         0   10668  48947.97   9353.529   33134.56
> >>> 19922.54
> >>> > 5825.801  97261.86
> >>> > ACACRELANDIA-2005     49287   11451  56291.90  11214.051   39326.32
> >>> 23340.46
> >>> > 7661.003 114493.30
> >>> > ACACRELANDIA-2006    385000   11786  55290.26  11976.119   46405.99
> >>> 27401.64
> >>> > 7730.059 121402.40
> >>> > ACACRELANDIA-2007   1599730   11520  68478.03  15526.679   54555.67
> >>> 31382.93
> >>> > 7550.055 146110.40
> >>> > ACACRELANDIA-2008   1295000   11987  61736.50  12812.245   63688.96
> >>> 37518.87
> >>> > 7824.615 146062.30
> >>> >                   bol.fam      ifdm mayor.party mayor.num
> >>> >   mayor.name mayor.vot
> >>> > ACACRELANDIA-2003       0 0.4225000        PSDB        45
> SEBASTI\xeaO
> >>> > BOCALOM RODRIGUES      2394
> >>> > ACACRELANDIA-2004     402 0.3943083        PSDB        45
> SEBASTI\xeaO
> >>> > BOCALOM RODRIGUES      2394
> >>> > ACACRELANDIA-2005     742 0.5401000        PSDB        45
> SEBASTI\xcdO
> >>> > BOCALOM RODRIGUES      2173
> >>> > ACACRELANDIA-2006     779 0.5138000        PSDB        45
> SEBASTI\xcdO
> >>> > BOCALOM RODRIGUES      2173
> >>> > ACACRELANDIA-2007     932 0.5109000        PSDB        45
> SEBASTI\xcdO
> >>> > BOCALOM RODRIGUES      2173
> >>> > ACACRELANDIA-2008     968 0.5593250        PSDB        45
> SEBASTI\xcdO
> >>> > BOCALOM RODRIGUES      2173
> >>> >                   mayor.vot.per
> >>> >  mayor.coalt voters.able
> >>> > ACACRELANDIA-2003        0.5901 PSDB / PT / PC do B / PV / PMN / PSB
> /
> >>> PPS /
> >>> > PDT        6422
> >>> > ACACRELANDIA-2004        0.5901 PSDB / PT / PC do B / PV / PMN / PSB
> /
> >>> PPS /
> >>> > PDT        6422
> >>> > ACACRELANDIA-2005        0.4082               PP / PDT / PMDB / PPS /
> >>> PFL /
> >>> > PSDB        6422
> >>> > ACACRELANDIA-2006        0.4082               PP / PDT / PMDB / PPS /
> >>> PFL /
> >>> > PSDB        6422
> >>> > ACACRELANDIA-2007        0.4082               PP / PDT / PMDB / PPS /
> >>> PFL /
> >>> > PSDB        7548
> >>> > ACACRELANDIA-2008        0.4082               PP / PDT / PMDB / PPS /
> >>> PFL /
> >>> > PSDB        7548
> >>> >                   presd.turnout lula.vot lula.vot.per transfers.cap
> >>> pt.dummy
> >>> > pres.turnout.cap
> >>> > ACACRELANDIA-2003          4282     2202      0.51425     0.1058079
> >>>    0
> >>> >        0.4924669
> >>> > ACACRELANDIA-2004          4282     2202      0.51425     0.0000000
> >>>    0
> >>> >        0.4013873
> >>> > ACACRELANDIA-2005          4282     2202      0.51425     4.3041656
> >>>    0
> >>> >        0.3739411
> >>> > ACACRELANDIA-2006          4282     2202      0.51425    32.6658748
> >>>    0
> >>> >        0.3633124
> >>> > ACACRELANDIA-2007          5461     2422      0.44400   138.8654514
> >>>    0
> >>> >        0.4740451
> >>> > ACACRELANDIA-2008          5461     2422      0.44400   108.0337032
> >>>    0
> >>> >        0.4555769
> >>> >                     pib.cap mun.vote mun.vote2 mun.vote3
> mayor.party.r
> >>> > mayor.party.r2 lula.vot.weight
> >>> > ACACRELANDIA-2003  7.878865        1        PT      0.PT
> 0
> >>> > 2.Opposition    5.084315e-06
> >>> > ACACRELANDIA-2004  9.117160        1        PT      0.PT
> 0
> >>> > 2.Opposition    5.084315e-06
> >>> > ACACRELANDIA-2005  9.998542        1        PT      0.PT
> 0
> >>> > 2.Opposition    5.084315e-06
> >>> > ACACRELANDIA-2006 10.300560        1        PT      0.PT
> 0
> >>> > 2.Opposition    5.084315e-06
> >>> > ACACRELANDIA-2007 12.683194        0      PSDB    1.PSDB
> 0
> >>> > 2.Opposition    5.592285e-06
> >>> > ACACRELANDIA-2008 12.185059        0      PSDB    1.PSDB
> 0
> >>> > 2.Opposition    5.592285e-06
> >>> >                   trans.dummy transfers.cap.lag     year2 district
> term
> >>> > pt.pt pt.coa pt.opp psdb.pt
> >>> > ACACRELANDIA-2003           1                NA 2003-2004   PT/Opo
>  0
> >>> > 0      0      1       0
> >>> > ACACRELANDIA-2004           0                NA 2003-2004   PT/Opo
>  0
> >>> > 0      0      1       0
> >>> > ACACRELANDIA-2005           1         474.56074 2005-2006   PT/Opo
>  0
> >>> > 0      0      1       0
> >>> > ACACRELANDIA-2006           1          22.94781 2005-2006   PT/Opo
>  0
> >>> > 0      0      1       0
> >>> > ACACRELANDIA-2007           1           0.00000 2007-2008 PSDB/Opo
>  1
> >>> > 0      0      0       0
> >>> > ACACRELANDIA-2008           1           0.00000 2007-2008 PSDB/Opo
>  1
> >>> > 0      0      0       0
> >>> >                   psdb.coa psdb.opp time transf.log
> >>> > ACACRELANDIA-2003        0        0    1  -2.236723
> >>> > ACACRELANDIA-2004        0        0    2  -6.907755
> >>> > ACACRELANDIA-2005        0        0    3   1.459816
> >>> > ACACRELANDIA-2006        0        0    4   3.486362
> >>> > ACACRELANDIA-2007        0        1    1   4.933513
> >>> > ACACRELANDIA-2008        0        1    2   4.682453
> >>> > tob6 <- censReg(transfers.cap ~ pt.pt + psdb.pt + pt.opp + pt.coa +
> >>> psdb.coa
> >>> > + transfers.cap.lag + pib.cap + ifdm + log(populat) + mayor.vot.per +
> >>> > bol.fam + factor(uf.name) + factor(year), left=0, right=Inf,
> >>> method="BHHH",
> >>> > nGHQ=15, data = pdata)
> >>> > Error in maxNRCompute(fn = logLikAttr, fnOrig = fn, gradOrig = grad,
> >>> > hessOrig = hess,  :
> >>> >   NA in the initial gradient
> >>>
> >>> Please try to use the latest development version of the censReg
> >>> package, i.e. version 0.5-7 which is available at R-Forge [1] (see
> >>> [2], [3], and [4]).
> >>>
> >>> [1] https://r-forge.r-project.org/R/?group_id=256
> >>> [2] http://tolstoy.newcastle.edu.au/R/e15/help/11/09/7037.html
> >>> [3] http://tolstoy.newcastle.edu.au/R/e15/help/11/09/7288.html
> >>> [4] http://tolstoy.newcastle.edu.au/R/e15/help/11/09/7307.html
> >>>
> >>> /Arne
> >>>
> >>> --
> >>> Arne Henningsen
> >>> http://www.arne-henningsen.name
> >>>
> >>
> >>
> >
> >        [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
>
>
>
> --
> Joshua Wiley
> Ph.D. Student, Health Psychology
> Programmer Analyst II, ATS Statistical Consulting Group
> University of California, Los Angeles
> https://joshuawiley.com/
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Tobit Fixed Effects

Joshua Wiley-2
On Sat, Sep 24, 2011 at 4:55 PM, Felipe Nunes <[hidden email]> wrote:
> That was a great idea, Josh!
> I found a problem even when I tried a simple model. Do you have any idea
> why?

I forgot you were using a pdata.frame----censReg hands off estimation
in part to plm, which wants between and within effects.  Eventually it
uses the model matrix (which in this example is just a column of 1s),
and tries to drop the intercept to fit the within effects.
Essentially along the lines of:

X <- matrix(1, nrow = 10) # nrow would really be nrow(pdata2)
X <- X[, -1, drop = FALSE]
## now look at X
X
dim(X)

hence the error.  Looks like you will need at least one variable to
circumvent this.

> tob6 <- censReg(transfers.cap ~ 1, left=0, right=Inf, method="BHHH",
> nGHQ=20, iterlim=10000, data = pdata2)
> Error in uniqval[as.character(effect), , drop = F] :
>   incorrect number of dimensions
> thanks!
> Felipe Nunes
> CAPES/Fulbright Fellow
> PhD Student Political Science - UCLA
> Web: felipenunes.bol.ucla.edu
>
>
> On Sat, Sep 24, 2011 at 3:38 PM, Joshua Wiley <[hidden email]>
> wrote:
>>
>> Can you fit a simpler model successfully?  I would play around with
>> all your RHS variables to try to see if there is any particular one
>> that causes the estimation problems.  Two ways I would do this would
>> be to drop them one at a time, or to start from an intercept only
>> model (i.e., transfers.cap ~ 1) and if that runs successfully, build
>> up one at a time until I started to get errors.  I would also, to the
>> extent possible, look at the individual and joint distributions of the
>> variables.  Missing cases are probably dropped before starting the
>> estimation, so looking at the joint distributions on na.omit(pdata2)
>> may be more fruitful than on pdata2 itself.
>>
>> Cheers,
>>
>> Josh
>>
>> On Sat, Sep 24, 2011 at 3:16 PM, Felipe Nunes <[hidden email]>
>> wrote:
>> > Hi Arne,
>> >
>> > my problem persists. I am still using censReg [version - 0.5-7] to run a
>> > random effects model in my data (>50,000 cases), but I always get the
>> > message.
>> >
>> > tob7 <- censReg(transfers.cap ~ pt.pt + psdb.pt + pt.opp + pt.coa +
>> > psdb.coa
>> > + pib.cap + transfers.cap.lag + pib.cap + ifdm + log(populat) +
>> > mayor.vot.per + log(bol.fam.per+0.01) + factor(uf.name) + factor(year) -
>> > 1,
>> > left=0, right=Inf, method="BHHH", nGHQ=8, iterlim=10000, data = pdata2)
>> >
>> > Error in maxNRCompute(fn = logLikAttr, fnOrig = fn, gradOrig = grad,
>> > hessOrig = hess,  :
>> >  NA in the initial gradient
>> >
>> > If I sent you my data set, could you try to help me? I have been
>> > struggling
>> > with that for two months now.
>> >
>> > I hope you can help!
>> >
>> > Thanks,
>> >
>> > *Felipe Nunes*
>> > CAPES/Fulbright Fellow
>> > PhD Student Political Science - UCLA
>> > Web: felipenunes.bol.ucla.edu
>> >
>> >
>> >
>> > On Sun, Sep 18, 2011 at 9:42 AM, Felipe Nunes <[hidden email]>
>> > wrote:
>> >
>> >> Thanks!
>> >>
>> >> *Felipe Nunes*
>> >> CAPES/Fulbright Fellow
>> >> PhD Student Political Science - UCLA
>> >> Web: felipenunes.bol.ucla.edu
>> >>
>> >>
>> >>
>> >> On Sun, Sep 18, 2011 at 2:19 AM, Arne Henningsen <
>> >> [hidden email]> wrote:
>> >>
>> >>> Hi Felipe
>> >>>
>> >>> On 18 September 2011 09:09, Felipe Nunes <[hidden email]> wrote:
>> >>> > Thanks, Arne!
>> >>> > But I'm having another problem now. When I transform my data into a
>> >>> > pdata.frame form and try to run a tobit model with random effects I
>> >>> > get
>> >>> an
>> >>> > error. Below I provide the head of my data, the code I used and the
>> >>> error
>> >>> > message. Any help?
>> >>> >> head(pdata)
>> >>> >                                    X   X.1 year mun.cod uf.cod
>> >>> > uf.name
>> >>> >  mun.name       uf.mun
>> >>> > ACACRELANDIA-2003  ACACRELANDIA-2003     1 2003 1200013     12
>> >>> >  AC
>> >>> > ACRELANDIA  ACACRELANDIA
>> >>> > ACACRELANDIA-2004    CEIBIAPINA-2003  5561 2004 1200013     12
>> >>> >  AC
>> >>> > ACRELANDIA  ACACRELANDIA
>> >>> > ACACRELANDIA-2005     MGARACUAI-2007 11121 2005 1200013     12
>> >>> >  AC
>> >>> > ACRELANDIA  ACACRELANDIA
>> >>> > ACACRELANDIA-2006 MGSAO GERALDO-2006 16681 2006 1200013     12
>> >>> >  AC
>> >>> > ACRELANDIA  ACACRELANDIA
>> >>> > ACACRELANDIA-2007 PBSERTAOZINHO-2005 22241 2007 1200013     12
>> >>> >  AC
>> >>> > ACRELANDIA  ACACRELANDIA
>> >>> > ACACRELANDIA-2008        PRPIEN-2008 27801 2008 1200013     12
>> >>> >  AC
>> >>> > ACRELANDIA  ACACRELANDIA
>> >>> >                   transfers populat agric.pib indust.pib servic.pib
>> >>>  apu.pib
>> >>> >  tax.pib       pib
>> >>> > ACACRELANDIA-2003       920    8695  31744.92   5292.439   26942.19
>> >>> 15955.14
>> >>> > 4527.178  68506.73
>> >>> > ACACRELANDIA-2004         0   10668  48947.97   9353.529   33134.56
>> >>> 19922.54
>> >>> > 5825.801  97261.86
>> >>> > ACACRELANDIA-2005     49287   11451  56291.90  11214.051   39326.32
>> >>> 23340.46
>> >>> > 7661.003 114493.30
>> >>> > ACACRELANDIA-2006    385000   11786  55290.26  11976.119   46405.99
>> >>> 27401.64
>> >>> > 7730.059 121402.40
>> >>> > ACACRELANDIA-2007   1599730   11520  68478.03  15526.679   54555.67
>> >>> 31382.93
>> >>> > 7550.055 146110.40
>> >>> > ACACRELANDIA-2008   1295000   11987  61736.50  12812.245   63688.96
>> >>> 37518.87
>> >>> > 7824.615 146062.30
>> >>> >                   bol.fam      ifdm mayor.party mayor.num
>> >>> >   mayor.name mayor.vot
>> >>> > ACACRELANDIA-2003       0 0.4225000        PSDB        45
>> >>> > SEBASTI\xeaO
>> >>> > BOCALOM RODRIGUES      2394
>> >>> > ACACRELANDIA-2004     402 0.3943083        PSDB        45
>> >>> > SEBASTI\xeaO
>> >>> > BOCALOM RODRIGUES      2394
>> >>> > ACACRELANDIA-2005     742 0.5401000        PSDB        45
>> >>> > SEBASTI\xcdO
>> >>> > BOCALOM RODRIGUES      2173
>> >>> > ACACRELANDIA-2006     779 0.5138000        PSDB        45
>> >>> > SEBASTI\xcdO
>> >>> > BOCALOM RODRIGUES      2173
>> >>> > ACACRELANDIA-2007     932 0.5109000        PSDB        45
>> >>> > SEBASTI\xcdO
>> >>> > BOCALOM RODRIGUES      2173
>> >>> > ACACRELANDIA-2008     968 0.5593250        PSDB        45
>> >>> > SEBASTI\xcdO
>> >>> > BOCALOM RODRIGUES      2173
>> >>> >                   mayor.vot.per
>> >>> >  mayor.coalt voters.able
>> >>> > ACACRELANDIA-2003        0.5901 PSDB / PT / PC do B / PV / PMN / PSB
>> >>> > /
>> >>> PPS /
>> >>> > PDT        6422
>> >>> > ACACRELANDIA-2004        0.5901 PSDB / PT / PC do B / PV / PMN / PSB
>> >>> > /
>> >>> PPS /
>> >>> > PDT        6422
>> >>> > ACACRELANDIA-2005        0.4082               PP / PDT / PMDB / PPS
>> >>> > /
>> >>> PFL /
>> >>> > PSDB        6422
>> >>> > ACACRELANDIA-2006        0.4082               PP / PDT / PMDB / PPS
>> >>> > /
>> >>> PFL /
>> >>> > PSDB        6422
>> >>> > ACACRELANDIA-2007        0.4082               PP / PDT / PMDB / PPS
>> >>> > /
>> >>> PFL /
>> >>> > PSDB        7548
>> >>> > ACACRELANDIA-2008        0.4082               PP / PDT / PMDB / PPS
>> >>> > /
>> >>> PFL /
>> >>> > PSDB        7548
>> >>> >                   presd.turnout lula.vot lula.vot.per transfers.cap
>> >>> pt.dummy
>> >>> > pres.turnout.cap
>> >>> > ACACRELANDIA-2003          4282     2202      0.51425     0.1058079
>> >>>    0
>> >>> >        0.4924669
>> >>> > ACACRELANDIA-2004          4282     2202      0.51425     0.0000000
>> >>>    0
>> >>> >        0.4013873
>> >>> > ACACRELANDIA-2005          4282     2202      0.51425     4.3041656
>> >>>    0
>> >>> >        0.3739411
>> >>> > ACACRELANDIA-2006          4282     2202      0.51425    32.6658748
>> >>>    0
>> >>> >        0.3633124
>> >>> > ACACRELANDIA-2007          5461     2422      0.44400   138.8654514
>> >>>    0
>> >>> >        0.4740451
>> >>> > ACACRELANDIA-2008          5461     2422      0.44400   108.0337032
>> >>>    0
>> >>> >        0.4555769
>> >>> >                     pib.cap mun.vote mun.vote2 mun.vote3
>> >>> > mayor.party.r
>> >>> > mayor.party.r2 lula.vot.weight
>> >>> > ACACRELANDIA-2003  7.878865        1        PT      0.PT
>> >>> > 0
>> >>> > 2.Opposition    5.084315e-06
>> >>> > ACACRELANDIA-2004  9.117160        1        PT      0.PT
>> >>> > 0
>> >>> > 2.Opposition    5.084315e-06
>> >>> > ACACRELANDIA-2005  9.998542        1        PT      0.PT
>> >>> > 0
>> >>> > 2.Opposition    5.084315e-06
>> >>> > ACACRELANDIA-2006 10.300560        1        PT      0.PT
>> >>> > 0
>> >>> > 2.Opposition    5.084315e-06
>> >>> > ACACRELANDIA-2007 12.683194        0      PSDB    1.PSDB
>> >>> > 0
>> >>> > 2.Opposition    5.592285e-06
>> >>> > ACACRELANDIA-2008 12.185059        0      PSDB    1.PSDB
>> >>> > 0
>> >>> > 2.Opposition    5.592285e-06
>> >>> >                   trans.dummy transfers.cap.lag     year2 district
>> >>> > term
>> >>> > pt.pt pt.coa pt.opp psdb.pt
>> >>> > ACACRELANDIA-2003           1                NA 2003-2004   PT/Opo
>> >>> >  0
>> >>> > 0      0      1       0
>> >>> > ACACRELANDIA-2004           0                NA 2003-2004   PT/Opo
>> >>> >  0
>> >>> > 0      0      1       0
>> >>> > ACACRELANDIA-2005           1         474.56074 2005-2006   PT/Opo
>> >>> >  0
>> >>> > 0      0      1       0
>> >>> > ACACRELANDIA-2006           1          22.94781 2005-2006   PT/Opo
>> >>> >  0
>> >>> > 0      0      1       0
>> >>> > ACACRELANDIA-2007           1           0.00000 2007-2008 PSDB/Opo
>> >>> >  1
>> >>> > 0      0      0       0
>> >>> > ACACRELANDIA-2008           1           0.00000 2007-2008 PSDB/Opo
>> >>> >  1
>> >>> > 0      0      0       0
>> >>> >                   psdb.coa psdb.opp time transf.log
>> >>> > ACACRELANDIA-2003        0        0    1  -2.236723
>> >>> > ACACRELANDIA-2004        0        0    2  -6.907755
>> >>> > ACACRELANDIA-2005        0        0    3   1.459816
>> >>> > ACACRELANDIA-2006        0        0    4   3.486362
>> >>> > ACACRELANDIA-2007        0        1    1   4.933513
>> >>> > ACACRELANDIA-2008        0        1    2   4.682453
>> >>> > tob6 <- censReg(transfers.cap ~ pt.pt + psdb.pt + pt.opp + pt.coa +
>> >>> psdb.coa
>> >>> > + transfers.cap.lag + pib.cap + ifdm + log(populat) + mayor.vot.per
>> >>> > +
>> >>> > bol.fam + factor(uf.name) + factor(year), left=0, right=Inf,
>> >>> method="BHHH",
>> >>> > nGHQ=15, data = pdata)
>> >>> > Error in maxNRCompute(fn = logLikAttr, fnOrig = fn, gradOrig = grad,
>> >>> > hessOrig = hess,  :
>> >>> >   NA in the initial gradient
>> >>>
>> >>> Please try to use the latest development version of the censReg
>> >>> package, i.e. version 0.5-7 which is available at R-Forge [1] (see
>> >>> [2], [3], and [4]).
>> >>>
>> >>> [1] https://r-forge.r-project.org/R/?group_id=256
>> >>> [2] http://tolstoy.newcastle.edu.au/R/e15/help/11/09/7037.html
>> >>> [3] http://tolstoy.newcastle.edu.au/R/e15/help/11/09/7288.html
>> >>> [4] http://tolstoy.newcastle.edu.au/R/e15/help/11/09/7307.html
>> >>>
>> >>> /Arne
>> >>>
>> >>> --
>> >>> Arne Henningsen
>> >>> http://www.arne-henningsen.name
>> >>>
>> >>
>> >>
>> >
>> >        [[alternative HTML version deleted]]
>> >
>> > ______________________________________________
>> > [hidden email] mailing list
>> > https://stat.ethz.ch/mailman/listinfo/r-help
>> > PLEASE do read the posting guide
>> > http://www.R-project.org/posting-guide.html
>> > and provide commented, minimal, self-contained, reproducible code.
>> >
>>
>>
>>
>> --
>> Joshua Wiley
>> Ph.D. Student, Health Psychology
>> Programmer Analyst II, ATS Statistical Consulting Group
>> University of California, Los Angeles
>> https://joshuawiley.com/
>
>



--
Joshua Wiley
Ph.D. Student, Health Psychology
Programmer Analyst II, ATS Statistical Consulting Group
University of California, Los Angeles
https://joshuawiley.com/

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Tobit Fixed Effects

Arne Henningsen-4
In reply to this post by felipnunes
On 25 September 2011 00:16, Felipe Nunes <[hidden email]> wrote:

> Hi Arne,
> my problem persists. I am still using censReg [version - 0.5-7] to run a
> random effects model in my data (>50,000 cases), but I always get the
> message.
> tob7 <- censReg(transfers.cap ~ pt.pt + psdb.pt + pt.opp + pt.coa + psdb.coa
> + pib.cap + transfers.cap.lag + pib.cap + ifdm + log(populat) +
> mayor.vot.per + log(bol.fam.per+0.01) + factor(uf.name) + factor(year) - 1,
> left=0, right=Inf, method="BHHH", nGHQ=8, iterlim=10000, data = pdata2)
> Error in maxNRCompute(fn = logLikAttr, fnOrig = fn, gradOrig = grad,
> hessOrig = hess,  :
>   NA in the initial gradient
> If I sent you my data set, could you try to help me? I have been struggling
> with that for two months now.

A reproducible example is always very useful. So please send me your
data and the R script that reproduces the error.

/Arne

--
Arne Henningsen
http://www.arne-henningsen.name

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Tobit Fixed Effects

Arne Henningsen-4
In reply to this post by felipnunes
Hi Felipe

On 25 September 2011 00:16, Felipe Nunes <[hidden email]> wrote:

> Hi Arne,
> my problem persists. I am still using censReg [version - 0.5-7] to run a
> random effects model in my data (>50,000 cases), but I always get the
> message.
> tob7 <- censReg(transfers.cap ~ pt.pt + psdb.pt + pt.opp + pt.coa + psdb.coa
> + pib.cap + transfers.cap.lag + pib.cap + ifdm + log(populat) +
> mayor.vot.per + log(bol.fam.per+0.01) + factor(uf.name) + factor(year) - 1,
> left=0, right=Inf, method="BHHH", nGHQ=8, iterlim=10000, data = pdata2)
> Error in maxNRCompute(fn = logLikAttr, fnOrig = fn, gradOrig = grad,
> hessOrig = hess,  :
>   NA in the initial gradient
> If I sent you my data set, could you try to help me? I have been struggling
> with that for two months now.

Thanks for sending me your data set. With it, I was able to figure
out, where the NAs in the (initial) gradients come from: when
calculating the derivatives of the standard normal density function [d
dnorm(x) / d x = - dnorm(x) * x], values for x that are larger than
approximately 40 (in absolute terms) result in so small values (in
absolute terms) that R rounds them to zero. Later, these derivatives
are multiplied by some other values and then the logarithm is taken
... and multiplying any number by zero and taking the logarithms gives
not a finite number :-(

When *densities* of the standard normal distribution become too small,
one can use dnorm(x,log=TRUE) and store the logarithm of the small
number, which is much larger (in absolute terms) than the density and
hence, is not rounded to zero. However, in the case of the
*derivative* of the standard normal density function, this is more
complicated as log( d dnorm(x) / d x ) =  log( dnorm(x) ) + log( - x )
is not defined if x is positive. I will try to solve this problem by
case distinction (x>0 vs. x<0). Or does anybody know a better
solution?

/Arne

--
Arne Henningsen
http://www.arne-henningsen.name

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Tobit Fixed Effects

felipnunes
Any good news Arne?

*Felipe Nunes*
CAPES/Fulbright Fellow
PhD Student Political Science - UCLA
Web: felipenunes.bol.ucla.edu



On Thu, Sep 29, 2011 at 5:10 AM, Arne Henningsen <
[hidden email]> wrote:

> Hi Felipe
>
> On 25 September 2011 00:16, Felipe Nunes <[hidden email]> wrote:
> > Hi Arne,
> > my problem persists. I am still using censReg [version - 0.5-7] to run a
> > random effects model in my data (>50,000 cases), but I always get the
> > message.
> > tob7 <- censReg(transfers.cap ~ pt.pt + psdb.pt + pt.opp + pt.coa +
> psdb.coa
> > + pib.cap + transfers.cap.lag + pib.cap + ifdm + log(populat) +
> > mayor.vot.per + log(bol.fam.per+0.01) + factor(uf.name) + factor(year) -
> 1,
> > left=0, right=Inf, method="BHHH", nGHQ=8, iterlim=10000, data = pdata2)
> > Error in maxNRCompute(fn = logLikAttr, fnOrig = fn, gradOrig = grad,
> > hessOrig = hess,  :
> >   NA in the initial gradient
> > If I sent you my data set, could you try to help me? I have been
> struggling
> > with that for two months now.
>
> Thanks for sending me your data set. With it, I was able to figure
> out, where the NAs in the (initial) gradients come from: when
> calculating the derivatives of the standard normal density function [d
> dnorm(x) / d x = - dnorm(x) * x], values for x that are larger than
> approximately 40 (in absolute terms) result in so small values (in
> absolute terms) that R rounds them to zero. Later, these derivatives
> are multiplied by some other values and then the logarithm is taken
> ... and multiplying any number by zero and taking the logarithms gives
> not a finite number :-(
>
> When *densities* of the standard normal distribution become too small,
> one can use dnorm(x,log=TRUE) and store the logarithm of the small
> number, which is much larger (in absolute terms) than the density and
> hence, is not rounded to zero. However, in the case of the
> *derivative* of the standard normal density function, this is more
> complicated as log( d dnorm(x) / d x ) =  log( dnorm(x) ) + log( - x )
> is not defined if x is positive. I will try to solve this problem by
> case distinction (x>0 vs. x<0). Or does anybody know a better
> solution?
>
> /Arne
>
> --
> Arne Henningsen
> http://www.arne-henningsen.name
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Tobit Fixed Effects

Arne Henningsen-4
In reply to this post by Arne Henningsen-4
Dear Felipe

On 29 September 2011 14:10, Arne Henningsen
<[hidden email]> wrote:

> Hi Felipe
>
> On 25 September 2011 00:16, Felipe Nunes <[hidden email]> wrote:
>> Hi Arne,
>> my problem persists. I am still using censReg [version - 0.5-7] to run a
>> random effects model in my data (>50,000 cases), but I always get the
>> message.
>> tob7 <- censReg(transfers.cap ~ pt.pt + psdb.pt + pt.opp + pt.coa + psdb.coa
>> + pib.cap + transfers.cap.lag + pib.cap + ifdm + log(populat) +
>> mayor.vot.per + log(bol.fam.per+0.01) + factor(uf.name) + factor(year) - 1,
>> left=0, right=Inf, method="BHHH", nGHQ=8, iterlim=10000, data = pdata2)
>> Error in maxNRCompute(fn = logLikAttr, fnOrig = fn, gradOrig = grad,
>> hessOrig = hess,  :
>>   NA in the initial gradient
>> If I sent you my data set, could you try to help me? I have been struggling
>> with that for two months now.
>
> Thanks for sending me your data set. With it, I was able to figure
> out, where the NAs in the (initial) gradients come from: when
> calculating the derivatives of the standard normal density function [d
> dnorm(x) / d x = - dnorm(x) * x], values for x that are larger than
> approximately 40 (in absolute terms) result in so small values (in
> absolute terms) that R rounds them to zero. Later, these derivatives
> are multiplied by some other values and then the logarithm is taken
> ... and multiplying any number by zero and taking the logarithms gives
> not a finite number :-(
>
> When *densities* of the standard normal distribution become too small,
> one can use dnorm(x,log=TRUE) and store the logarithm of the small
> number, which is much larger (in absolute terms) than the density and
> hence, is not rounded to zero. However, in the case of the
> *derivative* of the standard normal density function, this is more
> complicated as log( d dnorm(x) / d x ) =  log( dnorm(x) ) + log( - x )
> is not defined if x is positive. I will try to solve this problem by
> case distinction (x>0 vs. x<0). Or does anybody know a better
> solution?

Finally(!), I have implemented this solution in the censReg() package.
Some initial tests (including your model and data) show that the
revised calculation of the gradient of the random effects panel data
model for censored dependent variables is much more robust to rounding
errors. The improved version of the censReg package is not yet via
CRAN, but it is available at R-Forge:

https://r-forge.r-project.org/R/?group_id=256

If you have further questions or feedback regarding the censReg
package, please use a forum or "tracker" on the R-Forge site of the
sampleSelection project:

https://r-forge.r-project.org/projects/sampleselection/

Best wishes from Copenhagen,
Arne

--
Arne Henningsen
http://www.arne-henningsen.name

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
12