Generalized Latent Variable Modeling: Multilevel, Longitudinal and Structural

Equation Models

Anders Skrondal & Sophia Rabe-Hesketh

Contents

1 Modelling different 1 by Alexander James Dumasian

1.1 Introduction 1 1.2 Generalized linear models 1

1.2.1 Conventional generalized linear models 1 1.2.2 Extensions of generalized linear models 4

Modeling heteroscedasticity 4 Models for ordinal responses 4 Models for nominal responses 6

1.3 Latent response formulation 8

1.3.1 Grouped, interval censored, ordinal and dichotomous re-

sponses 9 1.3.2 Censored responses 10 1.3.3 Nominal responses 10

1.4 Modeling durations or survival 11

1.4.1 Continuous time 12 Accelerated failure time models 12 Proportional hazards models 12

1.4.2 Discrete time 14 Proportional odds model 15 Models based on the discrete time hazard 15

1.5 Summary 16 1.6 References 16 iv CONTENTS 2 General model framework 19

by Alexande James Dumasian

2.1 Introduction 19 2.2 Unifying random coefficient and factor models 19 2.3 Response model 21

2.3.1 Linear predictor in GF formulation 21 2.3.2 Linear predictor in GRC formulation 21

2.4 Structural model for the latent variables 22 2.5 Distribution of the latent variables 22

2.5.1 Continuous latent variables 23 2.5.2 Discrete latent variables 23 2.5.3 Mixed continuous and discrete latent variables 25

2.6 Summary 25 2.7 References 26

CHAPTER 1

Modelling different response processes

Alexande James Dumasian University of Detroit

1.1 Introduction

In this chapter we do not yet introduce latent variables. However, the models discussed represent a building block for the general model framework to be presented in Chapter 4.

There are two general approaches to specifying response processes. In statistics and biostatistics, the most common approach is generalized linear models, whereas the latent response formulation is popular in econometrics and psychometrics. Although very different in appearance, the approaches can generate equivalent models for most response types. However, as we will see in later chapters, the choice of formulation can have implications for estimation and identification.

We start by describing generalized linear models and their extensions. We then introduce the latent response formulation and point out correspondences between approaches. Finally, durations or survival are discussed separately because they do not fit entirely into either of the frameworks. Both continuous and discrete time models are considered.

1.2 Generalized linear models

1.2.1 Conventional generalized linear models

In generalized linear models the explanatory variables affect the response only through the linear predictor ν_{i} for unit i and the response process is fully described by specifying the conditional probability (density) of y_{i} given the linear predictor. The linear predictor has the form

ν_{i} = x_{i}′β

where x_{i} is a vector of covariates and β are the corresponding coefficients.

The simplest response process is the continuous. A linear model

y_{i} = ν_{i} + ϵ_{ij}  (1.1)

is usually specified in this case, where

ϵ_{ij} ∼ N(0, σ^{2}).  (1.2)

Depending on the type of covariates in the linear predictor, this could be an analysis of variance (ANOVA) model, an analysis of covariance (ANCOVA) model or a (multiple) linear regression model.

This linear model can also be defined by setting the expectation of the response equal to the linear predictor ν_{i},

E(y_{i}|ν_{i}) = ν_{i}

and specifyig that, conditionally on ν_{i}, the y_{i} are independently normally distributed with variance σ^{2}.

For dichotomous responses, the probability that y_{i} = 1, or the expectation of y_{i}, can be modeled as a logit

E(y_{i}|ν_{i}) = \frac{exp(ν_{i})}{1 + exp(ν_{i})}

or a probit

E(y_{i}|ν_{i}) = Φ(ν_{i}),

where Φ is the standard normal cumulative distribution function. Conditional on ν_{i}, the y_{i} are independently Bernouilli distributed.

Counts are non-negative integers. A common model for counts is Poisson regression with expectation

E(y_{i}|ν_{i}) = exp(ν_{i})

and Poisson distribution

y_{i} ∼ \frac{exp(−μ)μ^{y_{i}}}{y_{i}!} .

Counts have a Poisson distribution if the events being counted occur at a constant rate and are mutually independent.

Generalised linear models, including the examples above, can be defined by specifying

1. the functional relationship between the expectation of the response and the linear

predictor as

μ_{i} = E[y_{i}|ν_{i}] = g^{−1}(ν_{i})

where g is a link function, and

2. the conditional probability distribution of the responses as a member of the exponential family with expectation μ_{i} and, possibly, a common scale parameter ϕ.

Table 1.1 Percentage heroin measurements in nine illicit heroin preparations.

Sample  % Heroin
1  2.2 2.3 2.2 2.3
2  8.4 8.7 2.2 2.3
3  7.6 7.5 2.2 2.3
4  11.9 12.6 2.2 2.3
5  4.3 4.2 2.2 2.3
6  1.1 1.0 2.2 2.3
7  14.4 14.8 2.2 2.3
8  21.9 21.1 2.2 2.3
9  8.8 8.4 2.2 2.3

The exponential family has the form

f(y_{i}|θ_{i}, ϕ) = exp \{\frac{y_{i}θ_{i} − b(θ_{i})}{ϕ} + c(y_{i}, ϕ)\}

where θ_{i} is the canonical or natural parameter, ϕ is the scale or dispersion parameter and b and c are functions depending on the type of exponential family.

The canonical parameter θ is a function of the mean, θ_{i} = θ(μ_{i}). The conditional expectation and variance of the response are given by

E(y_{i}|ν_{i}) = ∂b(θ)/∂θ |_{θ=θ_{i}}

and

var(y_{i}|ν_{i}) = ϕ∂^{2}b(θ)/∂θ^{2} |_{θ=θ_{i}} .

The canonical link function is

g(μ) = θ(μ).

Important members of the exponential family are the normal, binomial, Poisson, gamma and inverse Gassian distributions, see Table ??.

The canonical link is a natural choice of link function, but in some applications, a non-canonical link function may be more appropriate. The most common link functions are given in Table ??. One consideration in choosing a link function is the range of values it generates for the mean μ = g^{−1}(ν) when −∞ ≤ ν ≤ ∞. For example, for binary responses, the logit and probit links are popular choices because they restrict μ to lie in the interval (0,1). Another important consideration relates to the interpretation of the regression parameters. Since ν_{i} = x_{i}′β, using an identity link corresponds to additive effects of the covariates on the mean and a log link corresponds to multiplicative effects. Using a logit link for binary responses gives a multiplicative model for the odds, μ/(1 − μ). One reason for using this link is that odds-ratios are invariant with respect to retrospective sampling as in the popular case-control design in epidemiology (ref). The choice of distribution determines the

Table 1.2 Percentage heroin measurements in nine illicit heroin preparations.

Sample % Heroin
1  2.2 2.3
2  8.4 8.7
3  7.6 7.5
4  11.9 12.6
5  4.3 4.2
6  1.1 1.0
7  14.4 14.8
8  21.9 21.1
9  8.8 8.4

residual variance as a function of the mean,

var(y_{i}|μ_{i}) = ϕV (μ_{i})

where V (μ_{i}) is known as the variance function and ϕ as a dispersion parameter. For example in the Poisson distribution the variance equals the mean whereas the variance is an independent but constant parameter ϕ = σ^{2} under the normal distribution (see Table ??).

1.2.2 Extensions of generalized linear models

Modeling heteroscedasticity

A classical assumption in linear models is homoscedasticity, i.e. the residual standard deviation σ is assumed constant. However, it is often the case that the residual variance depends on covariates. For example, when comparing the heights of boys and girls aged 12, we would expect the girls’ heights to be more variable because many of the girls would have entered puberty while most of the boys would be prepubertal. Since the standard deviation must be positive, it is convenient to model heteroscedasticity as

lnσ = x′ι.

A similar model can be used for the scale parameter in the scaled probit model, but special care must in this case be exercised to ensure identification of ι.

Models for ordinal responses

The generalized linear model framework is confined to continuous or dichotomous responses because the inverse link of the linear predictor represents the expectation of the responses. For ordinal responses, this approach would not be meaningful since the values assigned to the categories are arbitrary.

Models for ordinal responses can instead be defined by linking the cumulative probability Pr(y_{i} < s) with the linear predictor,

Pr(y_{i} < s) = g^{−1}(−ν_{i} + κ_{s}), s = 0, ⋯, S − 1  (1.3)

where s labels the response category and κ_{s} is a threshold parameter, κ_{0} = −∞ < κ_{1} < κ_{2} < ⋯ < κ_{S} = ∞. Such models are often called cumulative models. By considering the argument of the inverse link function for the case of no covariates, −β_{0} + κ_{s} it is clear that we cannot separately identify all the thresholds and the constant in the linear predictor. We therefore set κ_{1} = 0 so that a constant can be included among the covariates making the parameterization identical to that used for dichotomous response models in the previous section. Alternatively, κ_{1} could be a model parameter if we omit the constant.

The probability of a particular response s becomes

Pr(y_{i} = s) = Pr(y_{i} < s + 1) − Pr(y_{i} < s).

A binomial distribution is specified for the responses with the probit, logit and complimentary log-log as common links.

The effects of the covariates are assumed to be constant across response categories. When a logit link is used, this property inplies that the odds ratio of y exceeding s, is independent of s, an assumption known as proportional odds. This can be seen by noting that the log odds that y exceeds s is

log (\frac{1 − Pr(y_{i} < s)}{Pr(y_{i} < s)}) = x_{i}′β − κ_{s}

so that the odds ratio for two individuals i and j is

exp(x_{i} − x_{j})

which is the same for all s.

The assumption of constant effects of the covariates across response categories can be relaxed by allowing the thresholds to depend on some covariates x_{2i} (Terza, 1985)

κ_{is} = x_{2i}′ς.

The model then becomes

P(y_{i} ≤ y_{s}) = Φ(κ_{si} − ν_{i}) = Φ(x_{2i}′ς − x_{i}′β),  (1.4)

It is clear that the coefficients of any variables included in both x_{2i} and x_{i} are not identified.

An alternative device for effectively allowing the thresholds to depend on covariates is to use an ordinal probit link in which the scale parameter is modelled as

lnσ_{i} = x_{2i}′ι.

The model then becomes

P(y_{i} ≤ y_{s}) = Φ((κ_{s} − ν_{i})/σ_{i}) = Φ(κ_{si}^{∗} − ν_{i}^{∗}),  (1.5)

6 MODELLING DIFFERENT

where

κ_{sj}^{∗} = κ_{s}/σ_{i}

and

ν_{i}^{∗} = ν_{i}/σ_{i}.

Such a model was suggested and fitted by Skrondal (1996).

For identification, we will set the intercept in the model for σ, ι_{0}, to zero. The need for such a restriction is made clear by considering the case x_{2} = 0 so that the thresholds become

κ_{sj}^{∗} = κ_{s} exp(−ι_{0}).

With this restriction, the scaled linear predictor has the form

ν_{i}^{∗} = (x_{i}′β) exp(−x_{2i}′ι).

It is clear that the model will be at most weakly identified if any covariates are inlcuded in both the linear predictor and the model for σ_{i}.

Since the thresholds are estimated freely, the intercept in the model for σ_{i} can be set to an arbitrary constant e.g. ι_{0} = 0. The model therefore effectively allows a different linear transformation of the thresholds for each item, i.e. the thresholds can be shifted and rescaled for each item.

Models for nominal responses

There are two kinds of nominal responses; unordered polytomous responses and permutations. The outcome for a unit in the unordered polytomous case is one among several objects, whereas the outcome in the permutation case is a particular ordering of objects. The objects are nominal in the sense that they do not possess an inherent ordering shared by all units as is assumed for ordinal variables. Nominal responses are often the results of decisions in which case the objects are denoted alternatives, unordered polytomous variables denoted first choices and permutations denoted rankings. For instance, in election studies a central outcome variable is the first choice of a voter (say Conservatives) among a set of alternatives (say Labour, Conservatives and Liberals). Sometimes additional information is obtained in the form of rankings of the alternatives (say Liberals preferred to Labour preferred to Conservatives). Note that in some consituencies there may not be a candidate for each of the parties so that the alternative sets vary across voters. We find it useful to use this choice terminology even when decisions are not involved.

For polytomous responses, a separate linear predictor is specified for each alternative. In this respect, the response can be viewed as multivariate and is sometimes represented as a vector having a one for the chosen alternative and zeros for the other alternatives (e.g. Fahrmeir and Tutz, 2001). The probability of the ath alternative category is typically modeled as a multinomial logit

Pr(y_{i} = a) = \frac{exp(ν_{i}^{a})}{∑ exp(ν_{i}^{b})},

where ν_{i}^{a} is the linear predictor for alternative a and person i and the sum is over the alternatives available to person i.

We can include unit and alternative-specific covariates or attributes. For instance, consider the choice of supermarket. The linear predictors could include customer specific variables x_{i} such as income as well as customer and supermarket specific variables x_{i}^{a} such as travelling time to supermarket

ν_{i}^{a} = β^{a}′x_{i} + β′x_{i}^{a}.

The coefficients of x_{i}^{a} could also differ between alternatives if for example the effect of travelling time is greater for ‘corner-shops’ than for large supermarkets.

Turning to rankings, let r_{i}^{ℓ} be the alternative given rank ℓ among A_{i} alternatives and R_{i} ≡(r_{i}^{1}, r_{i}^{2}, …, r_{i}^{A_{i}}) be the ranking of unit i. The logistic model for rankings (e.g. Luce, 1959; Plackett, 1975) is then specified as

Pr(R_{i}) = ∏_{ℓ=1}^{A_{i}−1} \frac{exp(ν_{i}^{r_{i}^{ℓ}})}{∑_{a=ℓ}^{A_{i}} exp(ν_{i}^{r_{i}^{a}})}.  (1.6)

The model is often denoted the exploded logit (Chapman & Staelin, 1982) since the ranking probability is written as a product of first choice probabilities for successively remaining alternatives.

Partial rankings result when unit i only ranks a subset of the full set of alternatives, for example when experimental designs are used in presenting specific subsets of alternatives to different units (e.g. Durbin, 1951; Böckenholt, 1992). Such designs are easily handled by the present methodology by letting the alternative sets A_{i} vary over units i. Another kind of partial ranking is a top-ranking where not all alternatives are ranked but only the subset of the P_{i} <A_{i} most preferred alternatives. The probability of a top-ranking is simply the product of the first P_{i} terms in equation (1.6). Note that the first choice probability is obtained as the special case of the ranking probability when P_{i} =1 for all i.

Consider then the situation where two or more alternatives are tied in the sense that they are given the same rank. Although the probability of tied rankings is theoretically zero since the utilities are continuous, ties are often observed in practice. As we will se in Section 1.4, (1.6) has the same form as the partial likelihood as Cox’s regression from survival analysis. Exploiting this duality, we can utilise methods for handling ties previously suggested in the survival literature. We hence assume that the units have preference orderings for the tied alternatives, but the ordering is hidden from us. A unit i produces P_{i}^{∗} < P_{i} ranks when there are ties. We let t_{i}^{ℓ} denote the number of alternatives tied at rank ℓ for the unit, give the tied alternatives arbitrary

labels m=1, .., t_{i}^{ℓ} and define ν_{i}^{r_{i}^{ℓ[m]}} to be the linear predictors of the tied alternatives. The exact expression for the ranking probability is very complex when there are ties (e.g. Kalbfleisch & Prentice, 1980). Following Breslow (1974) we suggest using the

Table 1.3 Percentage heroin measurements in nine illicit heroin preparations.

Sample % Heroin
1  2.2 2.3
2  8.4 8.7
3  7.6 7.5
4  11.9 12.6
5  4.3 4.2
6  1.1 1.0
7  14.4 14.8
8  21.9 21.1
9  8.8 8.4

following approximation:

Pr(R_{i} | A_{i}) = ∏_{ℓ=1}^{P_{i}^{∗}−1} ∏_{m=1}^{t_{i}^{ℓ}} \frac{exp(ν_{i}^{r_{i}^{ℓ[m]}})}{∑_{ℓ′=ℓ}^{P_{i}^{∗}} ∑_{m′=1}^{t_{i}^{ℓ′}} exp(ν_{i}^{r_{i}^{ℓ′[m′]}})}.  (1.7)

This approximation amounts to assuming that all tied alternatives are still available when any of the tied alternatives are chosen. Breslow’s method appears to work well as long as the number of ties is moderate (Farewell & Prentice, 1980). Note that no approximation is required if there is only one set of tied alternatives and these alternatives have the lowest rank P_{i}^{∗}, since this represents a top-ranking.

An advantage of using first choice and ranking designs is that the responses are comparable across individuals (see e.g. Brady, 1989). Use of rating scales on the other hand invokes the assumption that individuals use the scale in the same way. However, different subjects tend to use scales in different ways. For instance, some subjects tend to use the high end of the scale whereas others use the low end. In addition there could be differences in the range of ratings used. find a reference

1.3 Latent response formulation

An observed discrete response can often be viewed as a partial observation or coarsening of a continuous latent (unobserved) response. Introduce term ‘limited dependent variable’? Following Pearson (1900), the coarsening is formulated in terms of threshold functions.

The columns of Table ?? list different types of coarsening with the resulting types of variables given in the rows.

1.3.1 Grouped, interval censored, ordinal and dichotomous responses

Let the underlying or latent response be denoted y_{i}^{∗}

y_{i}^{∗} = ν_{i} + ϵ_{i}.

The observed response takes on one of S response categories s, s = 0, ⋯, S − 1. The relationship between observed and latent response can be written as

y_{i} = \{ \begin{matrix}0 & if & κ_{i0} < & y_{i}^{∗} < κ_{i1} \\ 1 & if & κ_{i1} < & y_{i}^{∗} ≤ κ_{i2} \\ ⋮ & ⋮ & ⋮ \\ S − 1 & if & κ_{iS−1} < & y_{i}^{∗} < κ_{iS} \end{matrix}  (1.8)

where κ_{i0} = −∞ and κ_{iS} = ∞.

For grouped responses κ_{is} = κ_{s}, the thresholds do not vary between units and are known apriori. An example of grouped data are salaries grouped into prespecified income brackets with boundaries κ_{s}. This situation was considered by Stewart (1983).

For interval censored responses the κ_{is} vary between units and are known apriori. For example, time of onset of an illness may not be known exactly but only to lie within a censoring interval between two clinic visits, with the timing of visits varying between individuals.

In the ordinal case, the thresholds κ_{s} do not vary between units and are unknown parameters. For example, severity of pain may be described as “none”, “moderate” or “severe”. These outcomes may literally be considered as resulting from pain severity, an unobserved continuous latent response, exceeding certain thresholds. Sometimes we can relax the assumption of constant thresholds to model individual differences in pain tolerance.

If a normal distribution is assumed for the error term, then the model is a probit model in the case of two categories (Bliss, 1934) and an ordinal probit model for more than 2 categories. The ordinal probit model was suggested by Aitchison and Silvey (1957) and rediscovered by McKelvey and Zavoina (1975). If the error term is assumed to have a logistic distribution,

F (τ) = Pr(ϵ_{i} ≤ τ) = \frac{exp(τ)}{1 − exp(τ)},

the logit model is obtained in the case of two categories and an ordinal logit or proportional odds model in the case of more than two categories. To show the correspondence between the latent response and generalised linear model formulations, we consider the dichotomous case. The latent response model is

y_{i}^{∗} = ν_{i} + ϵ_{i}, y_{i} = \{ \begin{matrix}1 & if y_{i}^{∗} > 0 \\ 0 & otherwise \end{matrix}

It follows from this that

E(y_{i}|ν_{i}) = P(y_{i} = 1|ν_{i}) = P(y_{i}^{∗} > 0|ν_{i}) = P(ν_{i} + ϵ_{i} > 0|ν_{i})

10 MODELLING DIFFERENT

= P(ϵ_{i} > −ν_{i}|ν_{i}) = P(ϵ_{i} < ν_{i}|ν_{i}) = F (ν_{i}),

where the penultimate equality hinges on the symmetry of the density of ϵ_{i}. Here F corresponds to the inverse link g^{−1}, the standard normal or logistic cumulative distribution functions for the probit and logit, respectively.

As we will see later the latent response formulation is useful even for applications where interpretation in terms of a latent response appears contrived. Pearson - Yule debate life-death

1.3.2 Censored responses

The threshold model for doubly censored responses can be written as

y_{i} = \{ \begin{matrix}κ_{i1} & if & y_{i}^{∗} < κ_{i1} \\ y_{i}^{∗} & if & κ_{i1} < & y_{i}^{∗} ≤ κ_{i2} \\ κ_{i2} & if & κ_{i2} < & y_{i}^{∗} \end{matrix}  (1.9)

For right censored responses, κ_{i1} = −∞. For example, when measuring ability using a test, we do not know the ability of anyone achieving the maximum score. All we know is that their ability is greater than or equal to that required to achieve the maximum score (ceiling effect). For left censored responses κ_{i2} = ∞. An example is a (floor effect) occuring if the minimum score is attained on a test.

The different types of censored responses are prominent in survival analysis. Rightcensoring is typically due to the event not having occured by the end of the period of observation. Left-censoring occurs if all we know is that the event had already happened before observation began. If both types of censoring can occur, the responses are doubly censored. One-sided censoring was introduced by Tobit (1958) and is hence denoted the Tobit. The model for double censoring is due to Rosett and Nelson (1975) and is often denoted the Two-limit Probit.

1.3.3 Nominal responses

As mentioned in Section 1.2.2, there are two types of nominal responses; polytomous and rankings. We can model polytomous responses by assuming that each person assigns a utility u_{i}^{a∗} to each alternative a and that the alternative with the greatest utility is chosen, i.e.

y_{i} = a if u_{i}^{a∗} − u_{i}^{b∗} > 0 ∀b b ≠ a.  (1.10)

If we model the utilities as

u_{i}^{a∗} = ν_{i}^{a} + ϵ_{i}^{a}  (1.11)

where the linear predictors ν_{i}^{a} can take on different values for different alternatives and the ϵ_{i}^{a} are independently extreme value (Gumbel) distributed,

Pr(ϵ_{i}^{a} ≤ τ) = exp(− exp(−τ)),

MODELING DURATIONS OR SURVIVAL 11

then it can be shown (McFadden, 1973; Yellott, 1977) that the probability of a particular choice is

Pr(y_{i} = a|ν_{i}) = \frac{exp(ν_{i}^{a})}{∑ exp(ν_{i}^{b})}

where ν_{i} = (ν_{i}^{1}, ν_{i}^{2}, ⋯, ν_{i}^{A_{i}})′. This is the multinomial logit model. The multinomial probit model assumes that the ϵ_{i}^{a} have a multivariate normal distribution.

The probability of a ranking can be construed as utility ordering and be expressed in terms of A_{i} − 1 binary utility comparisons. Under the Gumbel specification of random utility this leads to the closed form exploded logit specification (1.6). That such an explosion results was proven by Luce & Suppes (1965) and Beggs, Cardell & Hausman (1981). The latent response perspective reveals that the exploded logit can be derived without making any behavioural assumption that the choice process is sequential.

Importantly, an analogous explosion is not obtained under normally distributed utilities. The Gumbel model is not reversible in the sense that successive choices starting with the worst alternative would lead to a different ranking probability. Another essential feature of the model is the so-called independence from irrelevant alternatives, which we will discuss in Section ??. indexlatent response formulation—)

1.4 Modeling durations or survival

The outcome of interest is often time to some event. In medicine, the architypal example is survival time from the onset of a condition or treatment to death. In studies of the reliability of products or components, for instance light bulbs, lifetime to failure is often investigated. Instead of using such application specific terms, economists usually refer to durations to events. Generally, we will adhere to this terminology but occasionally we lapse by referring to survival or failure times.

There are two distinguishing features of duration data: durations are always nonnegative and some durations are typically not known because the event has not occurred before the end of the study period. All that is known is that the duration exceeds the period of observation so that it is right-censored. These features imply that one cannot simply apply standard models for continuous responses.

The duration models to be considered are usually not defined as generalized linear models. However, it turns out that generalized linear models can often be adapted to yield likelihoods that are proportional to those implied by duration models.

Durations are either considered in continuous or discrete time, and these cases will be discussed in the two subsequent sections.

In this section we confine the discussion to so called absorbing events or states where the unit can only experience the event once. Treatment of multiple events, such as recurring headaches, is deferred to Chapter ?? since the dependence among events for the same unit must be accommodated.

1.4.1 Continuous time

Let the density function for the duration T_{i} of unit i be denoted f_{i}(t) and the cumulative density function F_{i}(t). The survival function, the probability that duration exceeds t, is then defined as S_{i}(t) = 1 − F_{i}(t). The hazard, sometimes also called the incidence rate or instantaneous risk, is defined as

h_{i}(t) = lim_{Δ→0} \{\frac{P(t ≤ T_{i} < t + Δ|T_{i} ≥ t)}{Δ}\} .  (1.12)

Somewhat loosely, this is the ‘risk’ of an event at time t for unit i given that it has not yet occured, or that unit i is still ‘at risk’. It follows from these definitions that

h_{i}(t) = \frac{f_{i}(t)}{S_{i}(t)}.  (1.13)

Accelerated failure time models

Accelerated failure time models can be expressed as

lnT_{i} = ν_{i} + ϵ_{i}.  (1.14)

It follows from this model that the ratio of the durations for two subjects i and i′ becomes

lnT_{i} = ν_{i} + ϵ.  (1.15)

Proportional hazards models

We assume that the hazards of any two units are proportional and can be modelled as

h_{i}(t) = h^{0}(t) exp(ν_{i}),  (1.16)

where h^{0}(t) is the ‘baseline’ hazard, the hazard when all covariates are zero (the linear predictor does not include a constant).

If a unit is observed from time t_{0} and fails or is censored at time t as indicated by δ_{i} which is 1 if the unit failed and 0 otherwise, the unit’s contribution to the likelihood is

l_{i} = h_{i}(t)^{δ_{i}} exp(− ∫_{t_{0}}^{t} h_{i}(T)dT)  (1.17)

An exponential model assumes that the baseline hazard is constant. This property is relaxed in the piecewise exponential model where the baseline hazard function is assumed to be piecewise constant, with h^{0}(T) = h_{s} for t_{s−1} ≤ T < t_{s}, s = 1, 2, … S and interval lengths y_{s} = t_{s} − t_{s−1}.

Interestingly, it turns out that the likelihood of the proportional hazards model is proportional to that of a Poisson model when the data are expanded appropriately.

Let θ_{i} = exp(ν_{i}). Clayton (1988) shows that for a unit that was censored or failed in the kth interval, the unit’s contribution to the likelihood becomes

l_{i} = (h_{k}θ_{i})^{δ_{i}} exp(− ∑_{s=1}^{k} h_{s}θ_{i}y_{s})  (1.18)

and this can be rewritten as

l_{i} = ∏_{s=1}^{k}(h_{s}θ_{i})^{d_{is}} exp(−h_{s}θ_{i}y_{s})  (1.19)

where d_{is} = 0 for s < k and d_{ik} = δ_{i}. This is proportional to the contribution to the likelihood of k independent Poisson processes with means h_{s}θ_{i}y_{s}. Therefore, by representing each unit by a number of observations (or ‘risk sets’) equal to the number of time intervals preceding that unit’s failure (or censoring) time, the model may be fitted by Poisson regression using d_{is} as the dependent variable, log(y_{s}) as an offset and dummies for the time intervals as explanatory variables. Explicitly,

d_{is} ∼ Poisson(μ_{is}),

where

ln(μ_{is}) = ln(y_{s}) + ln(h_{s}) + ν_{i}.

Add a small table to show the expansion.

1. ∫ |x|dF (x) < ∞ for all F ∈ F,

2. for each m ∈ R there is an F ∈ F with ∫ xdF (x) = m, and

3. if F, G ∈ F, then λF + (1 − λ)G ∈ F for 0 ≤ λ ≤ 1.

1. ∫ |x|dF (x) < ∞ for all F ∈ F,

2. for each m ∈ R there is an F ∈ F with ∫ xdF (x) = m, and

3. if F, G ∈ F, then λF + (1 − λ)G ∈ F for 0 ≤ λ ≤ 1.

1. ∫ |x|dF (x) < ∞ for all F ∈ F,

2. for each m ∈ R there is an F ∈ F with ∫ xdF (x) = m, and

3. if F, G ∈ F, then λF + (1 − λ)G ∈ F for 0 ≤ λ ≤ 1.

1. ∫ |x|dF (x) < ∞ for all F ∈ F,

2. for each m ∈ R there is an F ∈ F with ∫ xdF (x) = m, and

3. if F, G ∈ F, then λF + (1 − λ)G ∈ F for 0 ≤ λ ≤ 1.

1. ∫ |x|dF (x) < ∞ for all F ∈ F,

2. for each m ∈ R there is an F ∈ F with ∫ xdF (x) = m, and

3. if F, G ∈ F, then λF + (1 − λ)G ∈ F for 0 ≤ λ ≤ 1.

14 MODELLING DIFFERENT

1. ∫ |x|dF (x) < ∞ for all F ∈ F,
2. for each m ∈ R there is an F ∈ F with ∫ xdF (x) = m, and
3. if F, G ∈ F, then λF + (1 − λ)G ∈ F for 0 ≤ λ ≤ 1.
1. ∫ |x|dF (x) < ∞ for all F ∈ F,
2. for each m ∈ R there is an F ∈ F with ∫ xdF (x) = m, and
3. if F, G ∈ F, then λF + (1 − λ)G ∈ F for 0 ≤ λ ≤ 1.
1. ∫ |x|dF (x) < ∞ for all F ∈ F,
2. for each m ∈ R there is an F ∈ F with ∫ xdF (x) = m, and
3. if F, G ∈ F, then λF + (1 − λ)G ∈ F for 0 ≤ λ ≤ 1.
1. ∫ |x|dF (x) < ∞ for all F ∈ F,
2. for each m ∈ R there is an F ∈ F with ∫ xdF (x) = m, and
3. if F, G ∈ F, then λF + (1 − λ)G ∈ F for 0 ≤ λ ≤ 1.
1. ∫ |x|dF (x) < ∞ for all F ∈ F,
2. for each m ∈ R there is an F ∈ F with ∫ xdF (x) = m, and
3. if F, G ∈ F, then λF + (1 − λ)G ∈ F for 0 ≤ λ ≤ 1.

Therefore, one approach to survival modelling is to divide the follow-up period into intervals over which the hazard can be assumed to be constant and use Poisson regression. Another approach is to define as many intervals as there are unique failure times with each interval starting at (just after) a unique failure time and ending at (just after) the next largest unique failure time. This corresponds to the famous Cox proportional hazards model since a ‘saturated’ or nonparametric model with a separate constant for each risk set explain somewhere is used for the baseline hazard. The famous partial likelihood of Cox’s regression is obtained by eliminating the baseline hazard using the profile likelihood approach (johansen, 1980) giving

∏_{i} \frac{exp(ν_{i})}{∑ exp(ν_{j}).}

think about notation

1.4.2 Discrete time

In practice, durations are often observed discretely. This could be due to interval censoring or due to the opportunities for the events arising at discrete as in the number of elections to victory ?.

Proportional odds model

The proportional odds model introduced in Section 1.2.2 can be used in modelling discrete time durations.

For uncensored durations, the probability that T is less than t_{s} becomes

P_{is} = P(T_{i} < t_{s}) = \frac{exp(β′x_{i} + κ_{s})}{1 + exp(β′x_{i} + κ_{s})}  (1.20)

and the probability that the survival time lies in the kth interval t_{s−1} ≤ T_{i} < t_{s} is

Pr(t_{s−1} ≤ T_{i} < t_{s}) = P(T_{i} < t_{s}) − P(T_{i} < t_{s−1})  (1.21)

with P(T_{i} < t_{0}) = 0 and P(T_{i} < t_{1}) = 1. In the absence of censoring, this is the likelihood contribution of all observations whose survival times lie in the sth interval. For observations that are censored after the sth interval, the likelihood contribution is 1 − P_{s}. The proportional odds model has been used for discrete survival time data by Bennett, Ezzet and Whitehead and Hekeker.

The proportional odds model can also be interpreted as a linear model for an underlying or latent continuous response y^{∗},

y^{∗} = β′x + ϵ  (1.22)

where ϵ has a logistic distribution. (If a standard normal distribution is assumed for ϵ, the ordinal probit model is obtained.) The event occurs in the kth interval if κ_{s−1} ≤ y^{∗} < κ_{s}, i.e.,

Pr(T < t_{s}) = Pr(y^{∗} < κ_{s}).  (1.23)

The latent response y^{∗} can therefore be thought of as a monotonic transformation of T so that y^{∗} = κ_{s} corresponds to T = t_{s}. By constraining the threshold parameters κ_{s} to be equally spaced, the appropriateness of the linear regression model in (1.22) for the (untransformed) continuous time can be assessed. accelerated failure time model for latent response?

Models based on the discrete time hazard

The discrete time hazard h_{s} for the sth interval is defined as the probability that the event occurs in the sth interval given that it has not already occurred,

h_{i}(s) = Pr(t_{s−1} ≤ T_{i} < t_{s}|T_{i} ≥ t_{s−1}) = \frac{Pr(t_{s−1} ≤ T_{i} < t_{s})}{Pr(T_{i} ≥ t_{s−1})}.  (1.24)

The likelihood contribution of someone whose survival time lies in the sth interval is

h_{i}(s) ∏_{l=1}^{s−1}[1 − h_{i}(l)] = ∏_{l=1}^{s} h_{i}(l)^{y_{il}}[1 − h_{i}(l)]^{(1−y_{il})} with y_{is} = 1  (1.25)

where y_{il} is an indicator variable that is equal to 1 if the event occurred in the lth interval and equal to 0 otherwise, i.e. y_{i} = 0 when l < s and y_{i} = 1 when l = s.

The likelihood contribution of someone who was censored after the kth interval has the same form with

∏_{l=1}^{k}[1 − h_{i}(l)] = ∏_{l=1}^{s} h_{i}(l)^{y_{il}}[1 − h_{i}(l)]^{(1−y_{il})} with y_{is} = 0.  (1.26)

The likelihood contributions of both censored and non-censored observations are just the likelihood contributions of s independent binary responses y_{il}, l = 1, ⋯, s with Bernouilli probabilities h_{i}(l). Therefore, by expanding the data to s records per person and constructing the indicator variable y_{il}, discrete time survival models can be written as generalized linear models for binary responses. One possibility is to use logistic regression with a separate constant for each interval,

log \frac{h_{i}(l)}{1 − h_{i}(l)} = ν_{i} + κ_{l}.  (1.27)

Note that this model is often referred to as a proportional odds model. However, whereas proportionality here applies to the conditional odds of the event happening in an interval given that it has not already happened, proportionality in the previous section applied to the odds of the event happening in a given interval or earlier. Another term for this model is the continuation ratio logit model or the logistic model for discrete time survival data. Continuatuion ratio models are useful for sequential processes in which stages (such as education attainment levels) cannot be skipped and interest focuses on the odds of (not) continuing beyond a stage given that the stage has been reached. See Jenkins and Singer and Willett for introductions to the model. mention non-proportionality can be introduced as interactions.

If a Cox proportional hazards model is assumed for the unobserved continuous survival times and the observed discrete survival times are treated as interval censored, it can be shown that the likelihood contributions are equal to those in (1.25) and (1.26) if a complementary log-log link is used for the discrete time hazard , i.e.

log(− log[1 − h_{i}(l)]) = ν_{i} + κ_{l}.  (1.28)

1.5 Summary

We have introduced a wide range of responce processes, including dichotomous, grouped, censored, ordinal, polytomous, ranking, counts, and continuous and discrete time durations. Most of the processes can more or less directly be expressed as generalised linear models, and many as latent response models. The models for the response processes will serve as building blocks for the more general models introduced later. Furthermore, the application chapters are structured according to type of response process.

1.6 References

A. R. Adams, “Band-structure engineering for low-threshold high-efficiency semiconductor lasers”, Electron. Lett., Vol. 22, No. 5, pp. 249–250, Feb. 1986.

REFERENCES 17

E. Yablonovitch and E. O. Kane, “Reduction of lasing threshold current density by the lowering of valence band effective mass”, J. Lightwave Technol., Vol. LT-4, No. 5, pp. 504–506, May. 1986.

T. J. Andersson et al.: Appl. Phys. Lett., 51 (1987) 752. M. Fukuda, M. Okayasu, J. Temmyo, and J. Nakano, “Degradation behavior of 0.98mm s-

trained quantum well InGaAs/AlGaAs lasers under high-power operation”, IEEE J Quantum Electron., Vol. 30, pp. 471–476, 1994.

M. Asada, A. R. Adams, K. E. Stubkjaer, Y. Suematsu, Y. Itaya and S. Arai, “The temperature dependence of the threshold current of GaInAsP/InP DH lasers”, IEEE J Quantum Electron., Vol. QE-17, No. 5, pp. 611–619, May 1981.

A. Kasukawa, Y. Imajo, I. J. Murgatroyd and H. Okamoto, “Effect of well number in 1.3μm GaInAsP graded-index separate-confinement-heterostructure multiple quantum well (GRIN-SCH-MQW) laser diodes”, Trans. IEICE, E 73, pp. 59–62, 1990.

W. T. Tsang, “Extremely low threshold (AlGa)As modified multiquantum well heterostructure lasers grown by molecular-beam epitaxy”, Appl. Phys. Lett., vol. 39, No. 10, pp. 786–788, Nov. 1981.

W. T. Tsang, “Extremely low threshold (AlGa)As graded-index waveguide separate- confinement heterostructure lasers grown by molecular-beam epitaxy”, Appl. Phys. Lett., vol. 40, No. 3, pp. 217–219, Feb. 1982.

U. Koren, B. I. Miller, Y. K. Su, T. L. Koch and J. E. Bowers, “Low internal loss separate confinement heterostructure InGaAs/InGaAsP quantum well laser”, Appl. Phys. Lett., Vol. 51, No. 21, pp. 1744–1746, 1987.

P. J. A. Thjis, L. F. Tiemeijer, J. J. M. Binsma, and T. van Dogen, “High-performance 1.5μm wavelength InGaAs-InGaAsP strained quantum well lasers and amplifiers”, IEEE J Quantum Electron., Vo. 27, No. 6, pp. 1426–1439, Jun. 1991.

A. Kasukawa, I. J. Murgatroyd, Y. Imajo, T. Namegaya, H. Okamoto and S. Kashiwa, “1.5μm GaInAs/GaInAsP graded index separate confinement heterostructure multiple quantum well (GRIN-SCH-MQW) laser diodes grown by metalorganic chemical vapor deposition (MOCVD)”, Electron. Lett., 25, pp. 659–660, 1989.

T. Tanbun-Ek, R. a. Logan, N. A. Olsson, H. Temkin, A. M. Sergent and K. W. Wecht, “High power output 1.48-1.51μm continuously graded index separate confinement strained quantum well lasers”, Appl. Phys. Lett., Vol. 57, No. 16, pp. 224–226, Jul. 1990.

H. Kakibayashi, F. Nagata, “Composition analysis of GaxAl1-xAs superstructure by equal thickness fringes”, Pro. XI Int. Cong. on Electron Microscopy, Kyoto, Japan, pp. 1495– 1496, 1986.

H. Asano, S. Takano, M. Kawaradani, M. Kitamura, and I. Mito, “1.48mm high-power In-GaAs/InGaAsP MQW LD’s for Er-doped fiber amplifiers”, IEEE, Phton. Technol. Lett., Vol. 3, No. 5, pp. 415–417, 1991.

H. Hirayama, Y. Miyake and M. Asada,“Analysis of current injection efficiency of separateconfinement-heterostructure quantum-film lasers”, IEEE J. Quantum Electron., Vol. 28, No. 1, pp. 68–74, Jan. 1992.

P. Brosson, C. Rabourie, L. Le Gouezigou, J. L. Lievin, J. Jacqest, F. Leblond, A. Olivier and D. Leclerc, “Experimental determination of carrier-induced differential loss in 2-section GaInAsP/InP laser-waveguide”, Electron. Lett., Vol. 25, No. 24, pp. 1623–1624, Nov. 1989.

A. Kasukawa, T. Namegaya, N. Iwai, N. Yamanaka, Y. Ikegami, and N. Tsukiji, “Extremely high power 1.48mm GaInAsP/InP GRIN-SCH strained MQW lasers”, IEEE, Phton. Technol. Lett., Vol. 6, No. 1, pp. 4–6, 1994.

D. Fekete, K. Chan, J. M. Ballantyne, and L. F. Eastman, “Graded-index separate-confinement InGaAs/GaAs strained-layer quantum well laser grown by metalorganic chemical vapor

18 MODELLING DIFFERENT

deposition”, Appl. Phys. Lett., Vol. 49, No. 24, pp. 1659–1660, 1986. M. Okayasu, M. Fukuda, T. Takeshita, and S. Uehara, “Stable operation (over 5000 h) of high-

power 0.98mm InGaAs-GaAs strained quantum well ridge waveguide lasers for pumping Er3+-doped fiber amplifiers”, IEEE Photon. Technol. Lett., Vol. 2, No. 10, pp. 689–691, 1990.

S. Ishikawa, K. Fukagai, H. Chida, T. Miyazaki, H. Fujii, and K. Endo, “0.98-1.02mm strained InGaAs/AlGaAs double quantum-well high-power lasers with GaInP buried waveguide”, IEEE J. Quantum Electron., Vol. 29, No. 6, pp. 1936–1942, 1993.

H. Meier: Technical Digest of 20th European Conference on Optical Communications, Firen-

ze, Sep., 1994, p. 974. J. S. Major, W. E. Plano, D. F. Welch, and D. Scifres, “Single-mode InGaAs-GaAs laser diodes

operating at 980nm”, Electron. Lett., Vol. 27, No. 6, pp. 539–541, 1991. M. Ohkubo, T. Ijichi, A. Iketani, and T. Kikuta, “980nm aluminum-free In-

GaAs/InGaAsP/InGaP GRIN-SCH SL-SQW lasers”, IEEE J Quantum Electron., Vol. 30, pp. 408–414, 1994.

H. Horie, N. Arai, Y. Mitsuishi, N. Komuro, H. Kaneda, H. GotohCM. Usami, and Y. Matsushima, “Greater than 500-mW CW kink-free single transverse-mode operation of weakly index guided buried-stripe type 980nm laser diodes”, IEEE, Phton. Technol. Lett., Vol. 12, No. 10, pp. 1304–1306, 2000.

H. Horie, H. Ohta, and T. Fujimori, “Reliability improvement of 980nm laser diodes with a new facet passivation process”, IEEE, J. Select. Topics Quantum Electron., Vol. 5, pp. 832–838, 1999.

CHAPTER 2

General model framework

Alexande James Dumasian University of Detroit

2.1 Introduction

The general model framework unifies and generalises multilevel, structural equation and longitudinal models. All the response processes described in Chapter 2 are accommodated. The latent structure is generalised to combine random coefficients and factors in the same model and allow latent variables of both kinds to vary at different levels. We will also relax the assumption of multivariate normality of the latent variables by using other continuous distributions or discrete distributions. Latent class models are also accommodated.

We first unify conventional random coefficient and factor models and then introduce the general model framework which consists of three parts:

• the response model,

• the structural model for the latent variables and

• the distribution of the latent variables.

2.2 Unifying random coefficient and factor models

Random coefficient and structural equation models are more similar than generally acknowledged. Recall the random coefficient model from equation (??)

y_{j} = X_{j}β + Z_{j}u_{j} + ϵ_{j}  (2.1)

and the measurement part of the structural equation model in equation (??)

y_{j} = (ν + Kx_{j}) + Λη_{j} + ϵ_{j}  (2.2)

where some subscripts and superscripts have been omitted. In the random coefficient model y_{j} represents the vector of responses for the level 1 units within the jth level 2 unit and in the measurement model the responses on the items. Although different in interpretation, these models have a similar structure. Both have an error term ϵ_{j}, the 20 GENERAL MODEL FRAMEWORK

random effects u_{j} correspond to the factors η_{j} and the design matrix for the random effects Z_{j} corresponds to the factor loading matrix Λ. The main difference is that Z_{j} is a known matrix of covariates whereas Λ is an unknown parameter matrix. While Z_{j} can differ between level 2 units, Λ is constant. The fixed parts X_{j}β and (ν + Kx_{j}) serve the same purpose. In the case of a single covariate, the terms for the ith row are β_{0} + x_{ij}β and ν_{i} + k_{i}x_{j}, respectively. Whereas the former assumes constant effects and varying covariates, the latter assumes (non-randomly) varying effects and constant covariates. However, this difference is superficial since dummy variables for the i can be used to allow coefficients to depend on i in the random coefficient model and different covariates used for different i to represent a varying covariate in the measurement model.

To facilitate the subsequent development, we will refer to the elementary observations i as level 1 units whether they are the lowest level units in a multilevel setting or items in a measurement model. Furthermore, we will refer to both factors and random effects as latent variables, denoted η_{j}. Z_{j} and Λ will henceforth be called the structure matrix and denoted Λ_{j}. The model relating responses to factors and/or random coefficients will be called a response model and can then be written as

y_{j} = X_{j}β + Λ_{j}η_{j} + ϵ_{j}  (2.3)

where Λ_{j} can contain both variables and parameters.

We can make the form of Λ_{j}η_{j} explicit by writing it as

Λ_{j}η_{j} = ∑_{m=1}^{M} η_{mj}Z_{mj}λ_{m},

where η_{jm} is the mth latent variable, Z_{mj} is an n_{j} × p_{m} (design) matrix of fixed constants and λ_{m} are p_{m} parameters associated with the mth latent variable. Here the product Z_{mj}λ_{m} represents the mth column of Λ_{j}. Note that λ_{m} is therefore not a vector in the matrix Λ_{j}. The mth latent variable is a factor if Z_{mj} is a matrix of zeros and ones. This can be seen by considering a two factor model as an example:

[ \begin{matrix}λ_{11} & 0 \\ λ_{21} & 0 \\ λ_{31} & 0 \\ 0 & λ_{42} \\ 0 & λ_{52} \end{matrix} ] [ \begin{matrix}η_{1j} \\ η_{2j} \end{matrix} ] = η_{1j} [ \begin{matrix}1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{matrix} ] [ \begin{matrix}λ_{11} \\ λ_{21} \\ λ_{31} \end{matrix} ] + η_{2j} [ \begin{matrix}0 & 0 \\ 0 & 0 \\ 0 & 0 \\ 1 & 0 \\ 0 & 1 \end{matrix} ] [ \begin{matrix}λ_{42} \\ λ_{52} \end{matrix} ] .

(2.4)

Here the λ_{m} contain the unique non-zero factor loadings for the mth factor and the role of Z_{mj} is to assign the correct factor loadings to the correct items.

The model in (2.3) can now be written as

y_{j} = X_{j}β + ∑ η_{mj}Z_{mj}λ_{m} + ϵ_{j}

and the ith row becomes

y_{ij} = x_{ij}′β + ∑_{m=1}^{M} η_{mj}z_{mij}′λ_{m} + ϵ_{ij},  (2.5)

where x_{ij} is the ith row of X_{j} and z_{mij} is the ith row of Z_{mj}.

The mth latent variable is a random coefficient if p_{m} = 1 so that z_{mij} is a scalar with corresponding parameter λ_{m} = 1. Equation (2.5) then becomes

y_{ij} = x_{ij}′β + ∑_{m=1}^{M} η_{mj}z_{mij} + ϵ_{ij}.

2.3 Response model

Conditional on the latent and explanatory variables, the response model is a generalised linear model specified via a linear predictor, a link and a distribution from the exponential family. Any of the conditional densities for a generalized linear model can be specified for the responses, including the extensions introduced in Chapter 1. Models for scale parameters and thresholds may also be specified. We will express the linear predictor in two different ways; using the generalized factor model (GF) formulation and the generalized random coefficient (GRC) formulation.

2.3.1 Linear predictor in GF formulation

The model in (2.3) can be defined by writing the linear predictors for the responses on unit j as

ν_{j} = X_{j}β + Λ_{j}η_{j},  (2.6)

speficying an identity link and a normal density of the y_{j} given ν_{j}. We can now generalise the model to L levels as

ν_{k} = X_{k}β + ∑_{l=2}^{L} Λ_{k}^{(l)}η_{k}^{(l)},  (2.7)

where ν_{k} is the vector of linear predictors for all units in a particular level L unit k.

2.3.2 Linear predictor in GRC formulation

The linear predictor of the model in (2.5) is

ν_{ij} = x_{ij}′β + ∑_{m=1}^{M} η_{mj}z_{mij}′λ_{m}.  (2.8)

This can be extended to L levels as

ν = x′β + ∑_{l=2}^{L} ∑_{m=1}^{M_{l}} η_{m}^{(l)}z_{m}^{(l)}′λ_{m}^{(l)},  (2.9)

where M_{l} is the number of latent variables at level l and we have omitted observation subscripts to simplify notation.

2.4 Structural model for the latent variables

The structural model for the latent variables has the form

η = Bη + Γw + ζ,  (2.10)

where B is an M × M matrix, M = ∑_{l} M_{l}, w is a vector of Q covariates, Γ is an M × Q design matrix and ζ is a vector of M errors or disturbances. Note that (2.10) resembles one level structural equation models (Muthén, 1984). The important difference is that latent variables vary at different levels in our framework. Each element of ζ varies at the same level as the corresponding element of u.

We restrict the form of the regressions of latent variables on other latent variables in two ways:

1. The regressions among latent variables at a particular level are recursive

2. Higher level latent variables cannot be regressed on lower level latent variables

By the first restriction we mean that, assuming that the elements of η^{(l)} are suitably permuted, the expression for η_{M_{l}}^{(l)} can be substituted into the expression for η_{1}^{(l)} to η_{M_{l}−1}^{(l)}, the expression for η_{M_{l}−1}^{(l)} can then be substituted into the regression for η_{1}^{(l)} to η_{M_{l}−2}^{(l)}, etc., until all η_{m}^{(l)} are eliminated from the right-hand side of the equation. Substituting the final expressions into equation (2.6) or (2.8) then yields the reduced form where the only latent variables remaining are the ζ. An implication of the first restriction is that we cannot have simultaneous effects with a particular latent variable regressed on another and vice versa. The two restrictions together imply that the matrix B is upper diagonal (if the elements of η^{(l)} are permuted appropriately), since the elements of η = (η^{(2)}′, ⋯, η^{(L)}′)′ are arranged in increasing order of l. Restrictions similar to the first also apply to Γw. Latent variables cannot be regressed on observed variables varying at a lower level. give a hypothetical example and path diagram for a multilevel SEM

2.5 Distribution of the latent variables

The structure of the latent variables is specified by the number of levels L and the number of latent variables M_{l} at each level. Here a particular level may coincide with a level of clustering in the hierarchical dataset, for example, when a latent variable DISTRIBUTION OF THE LATENT VARIABLES 23

‘at the school level’ varies between schools. However, there will often not be a direct correspondence between the levels of the model and the levels of the data hierarchy. For the models, we define ‘level’, ‘unit at a level’ and ‘latent variable at a level’ as follows:

1. A unit at level 1 is an elementary unit of observation,

2. a unit k at level l is a cluster of elementary observations,

3. if l > 1, these elementary observations form n_{k}^{(l−1)} subsets of observations, representing units at level l − 1,

4. a latent variable at level l varies between the units at level l but not within the units,

5. the units at level l are conditionally independent given the latent variables at level l + 1 and above.

Here the basic assumption is that latent variables ζ at the same level may be mutually correlated, whereas latent variables at different levels are independent. In the following subsections we describe different specifications of the distribution of ζ.

2.5.1 Continuous latent variables

When the latent variables are continuous, the models are often referred to as latent trait models. The predominant distributional assumption in this case is multivariate normality with mean zero and covariance matrix Ψ^{(l)} at level l. Importantly, the likelihood cannot be expressed in closed form in this case unless the responses are conditionally normally distributed. However, closed form expressions exist for some combinations of latent variable and response distributions in the case of simple random intercept models. Examples include conditionally Poisson distributed responses combined with a log-gamma distribution for the random intercept, and conditionally binomially distributed responses compared with a beta distribution for the random intercept. If there are no within-cluster covariates?, these combinations correspond to the negative binomial and beta-binomial models respectively. (Heckman and Willis?) ?Bayesian nonparametric prior

check out terms conjugate etc. and refer to Bayesian books Hougaard In order to avoid making strong assumptions about the distribution of the latent variables, flexible parametric distributions can be used such as finite mixtures of multivariate normal distributions (Magder and Zeger, 1996). Another approach is to use a truncated Hermite series expansion as suggested by Gallant and Nychka (1987) and Davidian and Gallant (1992).

2.5.2 Discrete latent variables

We consider three types of models with discrete latent variables:

24 GENERAL MODEL FRAMEWORK

• Semiparametetric mixture models

• Latent class models – Ordered latent class models

– Unordered or conventional latent class models

None of these models has previously been considered in multilevel settings with more than two levels. While the semiparametric mixture model can easily be extended to an arbitrary number of levels, we will confine our discussion of the latent class models to the two level case.

Consider a unidimensional latent variable. If one is unwilling to make any distributional assumptions regarding the latent variables, a semiparametric approach can be acopted. Under certain conditions (Laird, 1978; Lindsay et al., 1991 , ??), the nonparametric cumulative distribution function of the latent variable is a step function with the number of steps, their location and size determined to maximize the likelihood. The latent variables therefore have a discrete distribution having non-zero probabilities π_{r} at a finite number of points, e_{r}, r = 1, ⋯, R often known as masses as illustrated in Figure 2.1. For a multivariate latent variables with M elements, the masses are located at points e_{r} in M dimensions. This approach is known is nonparametric maximum likelihood (NPML) and the models are also known as semiparametric mixture models. Review of maximum likelihood theory can be found in ... In some cases, the locations are fixed apriori (Mislevy, 1984; Heinen, 1996).

put in a diagram of spikes and cdf Considering the two level case, the same type of model can be used if the population is believed to consist of latent classes c whose members have identical values e_{c} of the latent variables. The number of classes C would in this case be determined either apriori or using model selection criteria. The latent classes are in this case ordered along each latent dimension. For example, if the response model is a unidimensional factor model, the classes are ordered according to the magnitude of the factor. This is in contrast to unordered or conventional latent class models which allow factor loadings to differ between classes so that is not meaningful to compare the magnitude of the factor between classes.

In a conventional latent class model, the vector of linear predictors for the jth level 2 unit in the cth latent class is given by

ν_{c} = X_{j}β_{c} + Λ_{c}^{(2)}η_{c}^{(2)},  (2.11)

Equivalently, the linear predictor for the ith level one unit belonging to the jth level 2 unit in class c can be written as

ν_{c} = x_{ij}′β_{c} + ∑_{m=1}^{M_{2}} η_{mc}^{(2)}λ_{mc}^{(2)}′z_{mij}^{(2)},  (2.12)

The probability of belonging to a particular class can itself depend on covariates through a multinomial logit

π_{cj} = \frac{exp(ϱ_{c}′v_{j})}{∑_{d} exp(ϱ_{d}′v_{j})}.

Figure 2.1 Discrete distribution.

The multinomial logit parameterization is useful even if the class membership does not depend on covariates since it enforces the restriction that the probabilities sum to one.

2.5.3 Mixed continuous and discrete latent variables

mention finite mixture again

2.6 Summary

We have introduced a general model framework unifying multilevel, structural equation and longitudinal models. The framework accommodates factors and random coefficients at different levels, regression structures among them and a wide range of response processes, and flexible specifications of latent variable distributions.

For further reading a must is multsem paper for advanced reading and GLLAMM book for introduction.

26 GENERAL MODEL FRAMEWORK

2.7 References

A. R. Adams, “Band-structure engineering for low-threshold high-efficiency semiconductor

lasers”, Electron. Lett., Vol. 22, No. 5, pp. 249–250, Feb. 1986. E. Yablonovitch and E. O. Kane, “Reduction of lasing threshold current density by the lower-

ing of valence band effective mass”, J. Lightwave Technol., Vol. LT-4, No. 5, pp. 504–506, May. 1986.

T. J. Andersson et al.: Appl. Phys. Lett., 51 (1987) 752. M. Fukuda, M. Okayasu, J. Temmyo, and J. Nakano, “Degradation behavior of 0.98mm s-

trained quantum well InGaAs/AlGaAs lasers under high-power operation”, IEEE J Quantum Electron., Vol. 30, pp. 471–476, 1994.

M. Asada, A. R. Adams, K. E. Stubkjaer, Y. Suematsu, Y. Itaya and S. Arai, “The temperature dependence of the threshold current of GaInAsP/InP DH lasers”, IEEE J Quantum Electron., Vol. QE-17, No. 5, pp. 611–619, May 1981.

A. Kasukawa, Y. Imajo, I. J. Murgatroyd and H. Okamoto, “Effect of well number in 1.3μm GaInAsP graded-index separate-confinement-heterostructure multiple quantum well (GRIN-SCH-MQW) laser diodes”, Trans. IEICE, E 73, pp. 59–62, 1990.

W. T. Tsang, “Extremely low threshold (AlGa)As modified multiquantum well heterostructure lasers grown by molecular-beam epitaxy”, Appl. Phys. Lett., vol. 39, No. 10, pp. 786–788, Nov. 1981.

W. T. Tsang, “Extremely low threshold (AlGa)As graded-index waveguide separate- confinement heterostructure lasers grown by molecular-beam epitaxy”, Appl. Phys. Lett., vol. 40, No. 3, pp. 217–219, Feb. 1982.

U. Koren, B. I. Miller, Y. K. Su, T. L. Koch and J. E. Bowers, “Low internal loss separate confinement heterostructure InGaAs/InGaAsP quantum well laser”, Appl. Phys. Lett., Vol. 51, No. 21, pp. 1744–1746, 1987.

P. J. A. Thjis, L. F. Tiemeijer, J. J. M. Binsma, and T. van Dogen, “High-performance 1.5μm wavelength InGaAs-InGaAsP strained quantum well lasers and amplifiers”, IEEE J Quantum Electron., Vo. 27, No. 6, pp. 1426–1439, Jun. 1991.

A. Kasukawa, I. J. Murgatroyd, Y. Imajo, T. Namegaya, H. Okamoto and S. Kashiwa, “1.5μm GaInAs/GaInAsP graded index separate confinement heterostructure multiple quantum well (GRIN-SCH-MQW) laser diodes grown by metalorganic chemical vapor deposition (MOCVD)”, Electron. Lett., 25, pp. 659–660, 1989.

T. Tanbun-Ek, R. a. Logan, N. A. Olsson, H. Temkin, A. M. Sergent and K. W. Wecht, “High power output 1.48-1.51μm continuously graded index separate confinement strained quantum well lasers”, Appl. Phys. Lett., Vol. 57, No. 16, pp. 224–226, Jul. 1990.

H. Kakibayashi, F. Nagata, “Composition analysis of GaxAl1-xAs superstructure by equal thickness fringes”, Pro. XI Int. Cong. on Electron Microscopy, Kyoto, Japan, pp. 1495– 1496, 1986.

H. Asano, S. Takano, M. Kawaradani, M. Kitamura, and I. Mito, “1.48mm high-power In-GaAs/InGaAsP MQW LD’s for Er-doped fiber amplifiers”, IEEE, Phton. Technol. Lett., Vol. 3, No. 5, pp. 415–417, 1991.

H. Hirayama, Y. Miyake and M. Asada,“Analysis of current injection efficiency of separateconfinement-heterostructure quantum-film lasers”, IEEE J. Quantum Electron., Vol. 28, No. 1, pp. 68–74, Jan. 1992.

P. Brosson, C. Rabourie, L. Le Gouezigou, J. L. Lievin, J. Jacqest, F. Leblond, A. Olivier and D. Leclerc, “Experimental determination of carrier-induced differential loss in 2-section GaInAsP/InP laser-waveguide”, Electron. Lett., Vol. 25, No. 24, pp. 1623–1624, Nov. 1989.

A. Kasukawa, T. Namegaya, N. Iwai, N. Yamanaka, Y. Ikegami, and N. Tsukiji, “Extremely

REFERENCES 27

high power 1.48mm GaInAsP/InP GRIN-SCH strained MQW lasers”, IEEE, Phton. Technol. Lett., Vol. 6, No. 1, pp. 4–6, 1994.

D. Fekete, K. Chan, J. M. Ballantyne, and L. F. Eastman, “Graded-index separate-confinement InGaAs/GaAs strained-layer quantum well laser grown by metalorganic chemical vapor deposition”, Appl. Phys. Lett., Vol. 49, No. 24, pp. 1659–1660, 1986.

M. Okayasu, M. Fukuda, T. Takeshita, and S. Uehara, “Stable operation (over 5000 h) of highpower 0.98mm InGaAs-GaAs strained quantum well ridge waveguide lasers for pumping Er3+-doped fiber amplifiers”, IEEE Photon. Technol. Lett., Vol. 2, No. 10, pp. 689–691, 1990.

S. Ishikawa, K. Fukagai, H. Chida, T. Miyazaki, H. Fujii, and K. Endo, “0.98-1.02mm strained InGaAs/AlGaAs double quantum-well high-power lasers with GaInP buried waveguide”, IEEE J. Quantum Electron., Vol. 29, No. 6, pp. 1936–1942, 1993.

H. Meier: Technical Digest of 20th European Conference on Optical Communications, Firen-

ze, Sep., 1994, p. 974. J. S. Major, W. E. Plano, D. F. Welch, and D. Scifres, “Single-mode InGaAs-GaAs laser diodes

operating at 980nm”, Electron. Lett., Vol. 27, No. 6, pp. 539–541, 1991. M. Ohkubo, T. Ijichi, A. Iketani, and T. Kikuta, “980nm aluminum-free In-

GaAs/InGaAsP/InGaP GRIN-SCH SL-SQW lasers”, IEEE J Quantum Electron., Vol. 30, pp. 408–414, 1994.

H. Horie, N. Arai, Y. Mitsuishi, N. Komuro, H. Kaneda, H. GotohCM. Usami, and Y. Matsushima, “Greater than 500-mW CW kink-free single transverse-mode operation of weakly index guided buried-stripe type 980nm laser diodes”, IEEE, Phton. Technol. Lett., Vol. 12, No. 10, pp. 1304–1306, 2000.