Question about bindata lib in high dimensions

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Question about bindata lib in high dimensions

Eman Ismail
I need to simulate data for 2000 binary variables given a vector of
marginal probabilities and a correlation matrix. I used bindata library,
but it give me

 Not all probabilities are between 0 and 1.
Error in Element ( i , j ): Admissible values are in [.....].
Error in commonprob2sigma(commonprob, simulvals) :
  Matrix commonprob not admissible.

 and I tried to get the elements within range but still have the same problem

How can I fix the correlation matrix or how to track error ??

Thanks
Eman

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Question about bindata lib in high dimensions

Eric Berger
Hi Eman,
It helps if you create a small example that reproduces the problem and then
post the code with your question.
This will help people determine what is causing the problem.

Best,
Eric


‪On Mon, Feb 11, 2019 at 11:52 AM ‫إيمان إسماعيل محمد‬‎ <
[hidden email]> wrote:‬

> I need to simulate data for 2000 binary variables given a vector of
> marginal probabilities and a correlation matrix. I used bindata library,
> but it give me
>
>  Not all probabilities are between 0 and 1.
> Error in Element ( i , j ): Admissible values are in [.....].
> Error in commonprob2sigma(commonprob, simulvals) :
>   Matrix commonprob not admissible.
>
>  and I tried to get the elements within range but still have the same
> problem
>
> How can I fix the correlation matrix or how to track error ??
>
> Thanks
> Eman
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Question about bindata lib in high dimensions

Eman Ismail
*Here Sample of Code for 10 variables:*
> probs_10 = probs[1:10]
> probs_10
 [1] 9.795272e-01 9.331778e-01 6.764349e-01 9.884067e-02 9.522222e-05
3.499417e-03 2.380556e-05 9.826457e-01 9.628633e-01 8.874949e-01
> corr_mat_10 = corr_mat[1:10,1:10]
> corr_mat_10
               [,1]         [,2]         [,3]         [,4]
[,5]          [,6]          [,7]          [,8]          [,9]       [,10]
 [1,]  1.0000000000  0.540258943  0.209031764  0.047879233 -6.750092e-02
0.0085672057  7.053822e-04  0.7840635867  0.6694665745  0.40604770
 [2,]  0.5402589429  1.000000000  0.386910326  0.088622750 -3.646798e-02
-0.0454879132  1.305637e-03  0.4929722619  0.6613106007  0.61159373
 [3,]  0.2090317635  0.386910326  1.000000000  0.229052428 -1.410984e-02
-0.0434598161 -7.054666e-03  0.1909793458  0.2831488805  0.49337866
 [4,]  0.0478792330  0.088622750  0.229052428  1.000000000 -3.231892e-03
-0.0101705338 -1.615888e-03  0.0434012259  0.0646190283  0.11766286
 [5,] -0.0675009217 -0.036467977 -0.014109837 -0.003231892  1.000000e+00
-0.0005782943 -4.761395e-05 -0.0734320072 -0.0496901947 -0.02740859
 [6,]  0.0085672057 -0.045487913 -0.043459816 -0.010170534 -5.782943e-04
1.0000000000  8.233515e-02  0.0078752345  0.0095061395 -0.03886223
 [7,]  0.0007053822  0.001305637 -0.007054666 -0.001615888 -4.761395e-05
0.0823351499  1.000000e+00  0.0006484086  0.0009582161  0.00173719
 [8,]  0.7840635867  0.492972262  0.190979346  0.043401226 -7.343201e-02
0.0078752345  6.484086e-04  1.0000000000  0.6766830516  0.37325133
 [9,]  0.6694665745  0.661310601  0.283148881  0.064619028 -4.969019e-02
0.0095061395  9.582161e-04  0.6766830516  1.0000000000  0.55158959
[10,]  0.4060477004  0.611593731  0.493378657  0.117662862 -2.740859e-02
-0.0388622278  1.737190e-03  0.3732513255  0.5515895878  1.00000000
> library(bindata)
> r <- rmvbin(10,margprob = probs_10, bincorr = corr_mat_10)
Not all probabilities are between 0 and 1.
Error in Element ( 1 , 5 ): Admissible values are in [ 0 ,
9.5222224867284e-05 ].
Error in Element ( 1 , 7 ): Admissible values are in [ 0 ,
2.3805556216821e-05 ].
Error in Element ( 3 , 7 ): Admissible values are in [ 0 ,
2.3805556216821e-05 ].
Error in Element ( 4 , 7 ): Admissible values are in [ 0 ,
2.3805556216821e-05 ].
Error in Element ( 5 , 7 ): Admissible values are in [ 0 ,
2.3805556216821e-05 ].
Error in Element ( 6 , 7 ): Admissible values are in [ 0 ,
2.3805556216821e-05 ].
Error in Element ( 7 , 3 ): Admissible values are in [ 0 ,
2.3805556216821e-05 ].
Error in Element ( 7 , 4 ): Admissible values are in [ 0 ,
2.3805556216821e-05 ].
Error in Element ( 7 , 5 ): Admissible values are in [ 0 ,
2.3805556216821e-05 ].
Error in Element ( 7 , 6 ): Admissible values are in [ 0 ,
2.3805556216821e-05 ].
Error in Element ( 7 , 8 ): Admissible values are in [ 0 ,
2.3805556216821e-05 ].
Error in Element ( 7 , 9 ): Admissible values are in [ 0 ,
2.3805556216821e-05 ].
Error in Element ( 7 , 10 ): Admissible values are in [ 0 ,
2.3805556216821e-05 ].
Error in Element ( 8 , 7 ): Admissible values are in [ 0 ,
2.3805556216821e-05 ].
Error in Element ( 9 , 7 ): Admissible values are in [ 0 ,
2.3805556216821e-05 ].
Error in Element ( 9 , 10 ): Admissible values are in [ 0.850358273621063 ,
0.887494941319304 ].
Error in commonprob2sigma(commonprob, simulvals) :
  Matrix commonprob not admissible.

*Then I tried to fix wrong values to be within range with the following:*
> corr_mat_10[1,5]=runif(1,min=0,max=9.5222224867284e-05)
> corr_mat_10[1,5]
[1] 7.915036e-05
> corr_mat_10[5,1]=7.915036e-05

*and did the same for all elements but the same error raised:*
> r <- rmvbin(10,margprob = probs_10, bincorr = corr_mat_10)
Error in Element ( 9 , 10 ): Admissible values are in [ 0.850358273621063 ,
0.887494941319304 ].
Error in commonprob2sigma(commonprob, simulvals) :
  Matrix commonprob not admissible.

*Finally I check the value of (9,10) it is within range but the same error
raised :*
> corr_mat_10[9,10]
[1] 0.8793437

I don't know How to fix it?

On Mon, 11 Feb 2019 at 12:23, Eric Berger <[hidden email]> wrote:

> Hi Eman,
> It helps if you create a small example that reproduces the problem and
> then post the code with your question.
> This will help people determine what is causing the problem.
>
> Best,
> Eric
>
>
> ‪On Mon, Feb 11, 2019 at 11:52 AM ‫إيمان إسماعيل محمد‬‎ <
> [hidden email]> wrote:‬
>
>> I need to simulate data for 2000 binary variables given a vector of
>> marginal probabilities and a correlation matrix. I used bindata library,
>> but it give me
>>
>>  Not all probabilities are between 0 and 1.
>> Error in Element ( i , j ): Admissible values are in [.....].
>> Error in commonprob2sigma(commonprob, simulvals) :
>>   Matrix commonprob not admissible.
>>
>>  and I tried to get the elements within range but still have the same
>> problem
>>
>> How can I fix the correlation matrix or how to track error ??
>>
>> Thanks
>> Eman
>>
>>         [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide
>> http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Question about bindata lib in high dimensions

Eman Ismail
even I tried to use another library mipfp to generate multivariate Bernoulli
*using the following:*
> p.joint <- ObtainMultBinaryDist(corr = corr_mat[1:10,1:10], marg.probs =
probs[1:10])
*it Shows:*
Problematic pairs:
     row col
[1,]  10   9
[2,]   9  10
Warning messages:
1: In Corr2PairProbs(corr, marg.probs) :
  Correlation exceeds constrains set by marg.probs, i.e. pair.proba[i, j]
<= marg.probs[i]

2: In Ipfp(seed = seed, target.list = target.list, target.data =
target.data,  :
  Missing values allowed in the target margins.
             Computation of the covariance matrices set to FALSE!
3: In Ipfp(seed = seed, target.list = target.list, target.data =
target.data,  :
  IPFP did not converged after 1000 iteration(s)!
            This migh be due to 0 cells in the seed, maximum number
            of iteration too low or tolerance too small

*and even if I fix the problematic pair (9,10) with the following:*

> corr_mat[9,10]=runif(1,max = min(probs[9],probs[10]),min  =
max(0,probs[10]+probs[9]-1))
> corr_mat[9,10]
[1] 0.8551618
> corr_mat[10,9]=0.8551618

*it still give me the same error.*

> p.joint <- ObtainMultBinaryDist(corr = corr_mat[1:10,1:10], marg.probs =
probs[1:10])
Problematic pairs:
     row col
[1,]  10   9
[2,]   9  10
Warning messages:
1: In Corr2PairProbs(corr, marg.probs) :
  Correlation exceeds constrains set by marg.probs, i.e. pair.proba[i, j]
<= marg.probs[i]

2: In Ipfp(seed = seed, target.list = target.list, target.data =
target.data,  :
  Missing values allowed in the target margins.
             Computation of the covariance matrices set to FALSE!
3: In Ipfp(seed = seed, target.list = target.list, target.data =
target.data,  :
  IPFP did not converged after 1000 iteration(s)!
            This migh be due to 0 cells in the seed, maximum number
            of iteration too low or tolerance too small
Could anyone help me please?

‪On Mon, 11 Feb 2019 at 13:10, ‫إيمان إسماعيل محمد‬‎ <
[hidden email]> wrote:‬

> *Here Sample of Code for 10 variables:*
> > probs_10 = probs[1:10]
> > probs_10
>  [1] 9.795272e-01 9.331778e-01 6.764349e-01 9.884067e-02 9.522222e-05
> 3.499417e-03 2.380556e-05 9.826457e-01 9.628633e-01 8.874949e-01
> > corr_mat_10 = corr_mat[1:10,1:10]
> > corr_mat_10
>                [,1]         [,2]         [,3]         [,4]
> [,5]          [,6]          [,7]          [,8]          [,9]       [,10]
>  [1,]  1.0000000000  0.540258943  0.209031764  0.047879233 -6.750092e-02
> 0.0085672057  7.053822e-04  0.7840635867  0.6694665745  0.40604770
>  [2,]  0.5402589429  1.000000000  0.386910326  0.088622750 -3.646798e-02
> -0.0454879132  1.305637e-03  0.4929722619  0.6613106007  0.61159373
>  [3,]  0.2090317635  0.386910326  1.000000000  0.229052428 -1.410984e-02
> -0.0434598161 -7.054666e-03  0.1909793458  0.2831488805  0.49337866
>  [4,]  0.0478792330  0.088622750  0.229052428  1.000000000 -3.231892e-03
> -0.0101705338 -1.615888e-03  0.0434012259  0.0646190283  0.11766286
>  [5,] -0.0675009217 -0.036467977 -0.014109837 -0.003231892  1.000000e+00
> -0.0005782943 -4.761395e-05 -0.0734320072 -0.0496901947 -0.02740859
>  [6,]  0.0085672057 -0.045487913 -0.043459816 -0.010170534 -5.782943e-04
> 1.0000000000  8.233515e-02  0.0078752345  0.0095061395 -0.03886223
>  [7,]  0.0007053822  0.001305637 -0.007054666 -0.001615888 -4.761395e-05
> 0.0823351499  1.000000e+00  0.0006484086  0.0009582161  0.00173719
>  [8,]  0.7840635867  0.492972262  0.190979346  0.043401226 -7.343201e-02
> 0.0078752345  6.484086e-04  1.0000000000  0.6766830516  0.37325133
>  [9,]  0.6694665745  0.661310601  0.283148881  0.064619028 -4.969019e-02
> 0.0095061395  9.582161e-04  0.6766830516  1.0000000000  0.55158959
> [10,]  0.4060477004  0.611593731  0.493378657  0.117662862 -2.740859e-02
> -0.0388622278  1.737190e-03  0.3732513255  0.5515895878  1.00000000
> > library(bindata)
> > r <- rmvbin(10,margprob = probs_10, bincorr = corr_mat_10)
> Not all probabilities are between 0 and 1.
> Error in Element ( 1 , 5 ): Admissible values are in [ 0 ,
> 9.5222224867284e-05 ].
> Error in Element ( 1 , 7 ): Admissible values are in [ 0 ,
> 2.3805556216821e-05 ].
> Error in Element ( 3 , 7 ): Admissible values are in [ 0 ,
> 2.3805556216821e-05 ].
> Error in Element ( 4 , 7 ): Admissible values are in [ 0 ,
> 2.3805556216821e-05 ].
> Error in Element ( 5 , 7 ): Admissible values are in [ 0 ,
> 2.3805556216821e-05 ].
> Error in Element ( 6 , 7 ): Admissible values are in [ 0 ,
> 2.3805556216821e-05 ].
> Error in Element ( 7 , 3 ): Admissible values are in [ 0 ,
> 2.3805556216821e-05 ].
> Error in Element ( 7 , 4 ): Admissible values are in [ 0 ,
> 2.3805556216821e-05 ].
> Error in Element ( 7 , 5 ): Admissible values are in [ 0 ,
> 2.3805556216821e-05 ].
> Error in Element ( 7 , 6 ): Admissible values are in [ 0 ,
> 2.3805556216821e-05 ].
> Error in Element ( 7 , 8 ): Admissible values are in [ 0 ,
> 2.3805556216821e-05 ].
> Error in Element ( 7 , 9 ): Admissible values are in [ 0 ,
> 2.3805556216821e-05 ].
> Error in Element ( 7 , 10 ): Admissible values are in [ 0 ,
> 2.3805556216821e-05 ].
> Error in Element ( 8 , 7 ): Admissible values are in [ 0 ,
> 2.3805556216821e-05 ].
> Error in Element ( 9 , 7 ): Admissible values are in [ 0 ,
> 2.3805556216821e-05 ].
> Error in Element ( 9 , 10 ): Admissible values are in [ 0.850358273621063
> , 0.887494941319304 ].
> Error in commonprob2sigma(commonprob, simulvals) :
>   Matrix commonprob not admissible.
>
> *Then I tried to fix wrong values to be within range with the following:*
> > corr_mat_10[1,5]=runif(1,min=0,max=9.5222224867284e-05)
> > corr_mat_10[1,5]
> [1] 7.915036e-05
> > corr_mat_10[5,1]=7.915036e-05
>
> *and did the same for all elements but the same error raised:*
> > r <- rmvbin(10,margprob = probs_10, bincorr = corr_mat_10)
> Error in Element ( 9 , 10 ): Admissible values are in [ 0.850358273621063
> , 0.887494941319304 ].
> Error in commonprob2sigma(commonprob, simulvals) :
>   Matrix commonprob not admissible.
>
> *Finally I check the value of (9,10) it is within range but the same error
> raised :*
> > corr_mat_10[9,10]
> [1] 0.8793437
>
> I don't know How to fix it?
>
> On Mon, 11 Feb 2019 at 12:23, Eric Berger <[hidden email]> wrote:
>
>> Hi Eman,
>> It helps if you create a small example that reproduces the problem and
>> then post the code with your question.
>> This will help people determine what is causing the problem.
>>
>> Best,
>> Eric
>>
>>
>> ‪On Mon, Feb 11, 2019 at 11:52 AM ‫إيمان إسماعيل محمد‬‎ <
>> [hidden email]> wrote:‬
>>
>>> I need to simulate data for 2000 binary variables given a vector of
>>> marginal probabilities and a correlation matrix. I used bindata library,
>>> but it give me
>>>
>>>  Not all probabilities are between 0 and 1.
>>> Error in Element ( i , j ): Admissible values are in [.....].
>>> Error in commonprob2sigma(commonprob, simulvals) :
>>>   Matrix commonprob not admissible.
>>>
>>>  and I tried to get the elements within range but still have the same
>>> problem
>>>
>>> How can I fix the correlation matrix or how to track error ??
>>>
>>> Thanks
>>> Eman
>>>
>>>         [[alternative HTML version deleted]]
>>>
>>> ______________________________________________
>>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>>
>>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.