Methods and Models Using Facility Data

The third step in the process involved analyses of the data compiled previously to test different methods for which pertinent data currently exist. Part of this process involved experimentation with different equations and computational methods to determine which specific formulas are most appropriate for each of the four types of facilities. These activities revealed a number of interesting and important insights about nursing shortages, which are summarized below.

A. Preliminary Analyses

Figure 8 presents the distribution of the indicator of difficulty recruiting RNs based on all facilities in North Carolina. The figure shows the number of facilities that experienced difficulty recruiting RNs (indicator >3) was more than double the number of the facilities with no difficulty recruiting RNs (indicator <3). In this case, 68 facilities (20.9%) reported not having difficulty recruiting RNs compared to 155 facilities (47.7%) that reported having difficulty recruiting RNs. The figure also shows that only 17 facilities (5.2%) reported that it was very easy to recruit RNs, in contrast to 56 facilities (17.2%) that reported it was very difficult to recruit RNs.

Figure 8. Distribution of RN Recruitment Difficulty Indicator, Based on Four Types of Health Facilities in North Carolina in 2004

[D]

Figure 9 presents the distribution of difficulty indicator by facility type. From this figure we can see that the distributions of difficulty to recruit RNs were different among all four types of facilities. For example, 4.6% of hospitals reported it was very difficult to recruit RNs, in contrast to 26.4% of public health facilities reported very difficult to recruit RNs.

Figure 9. Nursing Recruitment Difficulty Indicators in North Carolina, by Facility Type, 2004

[D]

Figure 10 compares the distributions of the predicted recruiting difficulty scores for the four types of facilities in North Carolina, based on the Ordered Probit model estimated using data for 2004. The figure shows clearly that the variation in recruiting difficulty is greatest for public health agencies and least for hospitals. It also shows that on average both public health agencies and long-term care facilities have statistically significantly greater difficulty recruiting RNs than hospitals (p≤0.05, since the 95% confidence intervals do not overlap).

Figure 10. Distribution of Predicted Difficulty Recruiting RNs in North Carolina by Type of Facility, 2004

[D]

Table 2 presents the distribution of facility type by difficulty indicator and Chi-Square statistic to test the null hypothesis that there is no association between type of facility and the difficulty recruiting RNs. Based on the Chi-square statistic, the null hypothesis was rejected (p = 0.011) because different types of facilities had different levels of difficulty recruiting RNs. The implication was that different types of facilities have different behaviors in term of modeling nursing shortages.

Chi-Square = 25.9 (df = 12)
Test of H₀: No association between type of facility and difficulty to recruit
H₀ is rejected with p-value = 0.011

Table 3 presents the distribution of difficulty indicator by number of adverse consequences of shortages and the Spearman correlation coefficient to test the null hypothesis that there is no relationship between difficulty indicator and number of consequences. From the Spearman correlation statistic, the null hypothesis was rejected (p<0.0005), meaning that on average facilities that experienced greater difficulty recruiting RNs had more bad consequences.

 Spearman correlation coefficient = 0.343
 Test of H₀: Correlation = 0
 H₀ is rejected with p-value < 0.0005

A number of models were estimated for hospitals in North Carolina. The steps followed are summarized below.

The indicator of nursing shortage used as a dependent variable was the number of reported negative effects on operations revealed by a facility. Most facilities indicated no effects or only one effect. The mean value for all facilities was 0.89, with a standard deviation of 1.07. Based on this, we defined facilities as being needy (for test purposes only), if they presented two or more effects on operations. Under this definition, 15.5% of hospitals were needy.

The population was adjusted by gender and age based on average use of primary care. This weighted older adults and infants more heavily than younger people and weighted women more heavily than men. The resulting variable was an estimate of how many primary care visits the population would require in a year’s time. Although the relationship between use of primary care and need for services, such as home health or long-term care, is open to debate, this variable was simply a way of standardizing the population based on characteristics known to affect medical need.

The following variables were selected for use in the North Carolina analyses:

1. Active RNs Employed in the County per 100,000 Adjusted Population
2. Students Enrolled in RN Programs in the County per 100,000 Adjusted Population
3. Number of Short-Term General Hospitals
4. Number of Short-Term General Hospital Beds
5. Ratio of Average RN Salary to Median Income
6. Number of Nursing and Personal Care Facilities
7. Percent of Population with Income Below Poverty Level
8. Population per Square Mile
9. Ratio of RNs to Hospital Beds
10. Number of Hours per Week Paid for Agency RNs
11. Number of Overtime RN Hours per Week
12. RN Vacancy Rate
13. RN Turnover Rate
14. Ratio of LPNs to RNs
15. Total Number of Budgeted RN Positions
16. Percent Non-Hispanic White

Average values for these variables are shown in Table 4 for three groups of hospitals in North Carolina.

Population per square mile was very highly correlated with several other variables, so a natural log transformation was applied to reduce problems of multicollinearity. There was also potential multicollinearity between the number of RNs per 100,000 adjusted population and number of general hospital beds per 100,000 adjusted population. Number of hospital beds was dropped in favor of number of hospitals.

4. Run OLS Regression Model, Full and Abbreviated

Two different OLS models were estimated to predict the number of adverse effects in hospitals in North Carolina, one for the full model that included both community and facility data and one that included only community data. These models are summarized below.

Full model

Table 5. Coefficients for Full OLS Regression Model to Predict Number of Adverse Effects of Nursing Shortages in Hospitals in North Carolina

Explanatory (Independent) Variable	Unstandardized Coefficients		Standardized Coefficients	t	p Value
	B	Std Err
Constant	-0.683	3.020	-	-0.226	0.822
RNs per 100,000 Adjusted Need	-0.0035	0.002	-0.353	-2.002	0.052
RN Salary to Average Salary	0.518	0.707	0.132	0.732	0.468
# Nursing and Personal Care Facilities	0.032	0.015	0.663	2.176	0.035
% Population Below Poverty, 2000	0.078	0.065	0.308	1.202	0.236
RNs per Hospital Bed	0.265	0.445	0.082	0.596	0.555
Hours of Agency RNs	0.0025	0.043	0.008	0.058	0.954
Hours of RN Overtime	-0.0008	0.016	-0.007	-0.052	0.959
RN Vacancy Rate	0.032	0.032	0.142	0.985	0.330
RN Turnover Rate	0.011	0.021	0.077	0.505	0.616
Persons per Square Mile (natural ln)	0.156	0.358	0.146	0.436	0.665
# Short-term Community Hospitals, ‘01	-0.359	0.134	-0.610	-2.690	0.010
RN Students per 100K Adjusted Need	-0.010	0.004	-0.392	-2.828	0.007
% Population Non-Hispanic White, 2004	-0.011	0.012	-0.167	-0.902	0.372

Dependent Variable: NUM_CONS
Selecting only cases for which FAC_TYPE = hospital
R² = 0.429
Abbreviated model

Because most of the variables that appeared most critical were community variables rather than facility variables, an abbreviated model was also run using only community information. Due to the constraints of data availability, the abbreviated model is one that can be used more easily in practice. The R², however, dropped substantially, from 0.429 in the full model to only 0.177 in the abbreviated model.

Table 6. Coefficients for Abbreviated OLS Regression Model to Predict Number of Adverse Effects of Nursing Shortages in Hospitals in North Carolina

Independent Variables	Unstandardized Coefficients		Standardized Coefficients	t	p Value
	Coefficient	Std Err
Constant	1.295	2.374	-	0.546	0.587
RNs per 100,000 Adjusted Need	-0.0012	0.001	-0.132	-0.880	0.382
RN Salary to Average Salary	0.281	0.582	0.081	0.482	0.631
# Nursing/Personal Care Facilities	0.023	0.012	0.494	1.905	0.061
% Population Below Poverty, 2000	0.033	0.053	0.136	0.622	0.536
RNs per Hospital Bed	0.044	0.402	0.013	0.108	0.914
Persons per Square Mile (natural ln)	-0.158	0.271	-0.159	-0.582	0.563
# Short-term Community Hospitals, ‘01	-0.227	0.109	-0.414	-2.076	0.041
RN Students per 100K Adjusted Need	-0.0054	0.003	-0.226	-1.967	0.053
% Population White Non-Hispanic, ‘04	-0.0053	0.010	-0.086	-0.511	0.611

Dependent Variable: NUM_CONS
Selecting only cases for which FAC_TYPE = hospital
R² = 0.177

Coefficients from the full and abbreviated regression models were used to estimate predicted number of problems in each facility. The top 16% of facilities in regard to predicted number of problems were considered to have made the test “cut” of 15.5% chosen arbitrarily based on earlier analysis (see Step 1). The facilities selected by the full model and the abbreviated model were compared to the facilities whose actual problem scores were in the top 15.5%.

Using the abbreviated model, 84% of hospitals were classified correctly based on the arbitrary value chosen earlier. Eight percent of facilities were misclassified as not needy by the abbreviated model when their actual scores qualified them as needy, while 7% were misclassified as being needy when their actual scores did not qualify them as such.

Using all the information in the full model would have increased the accuracy of prediction to 89%, with 5% of facilities erroneously classified as needy and 5% erroneously classified as not needy.

Using the information from the testing in Step 5, we conclude that using an abbreviated model with widely available community level data to assign facilities need scores would result in approximately 84% of facilities being correctly classified. Supplementing this with an appeals process requiring the additional information needed for the full model would correctly classify an additional 5% of facilities.

C. Empirical Models for North Dakota Hospitals

The coefficients estimated for North Carolina hospitals were applied to hospitals in North Dakota. The results are summarized below.

When the coefficients for the abbreviated model obtained from the empirical models developed for North Carolina were applied to hospitals in North Dakota, not surprisingly the classifications were less accurate. Seventy-nine percent of North Dakota hospitals were correctly classified by this application of North Carolina data, while 10% were erroneously classified as needy and 10% were erroneously classified as not needy.

This analysis suggests that using coefficients based on models estimated in one state achieves lower accuracy when applied to facilities in another state. Additional research would be required to determine whether the decline in accuracy might be related to the extent to which general characteristics of the states are similar or different.

D. Empirical Models for North Carolina Nursing Homes

The empirical models for nursing homes in North Carolina are summarized below.

The indicator of nursing shortage used as a dependent variable was the number of reported effects on operations reported by a facility. Most facilities reported no effects or only one effect. The mean value for all facilities was 1.0, with a standard deviation of 1.1. Based on this, we defined facilities as being needy (for test purposes only) if they reported two or more effects on operations. Under this definition, 31.3% of nursing homes were needy.

The population was adjusted by gender and age based on average use of primary care. This weighted older adults and infants more heavily than younger people and women more heavily than men. The resulting variable was an estimate of how many primary care visits the population would require in a year’s time. Although the relationship between use of primary care and need for services such as home health or long-term care is open to debate, this variable was simply a way of standardizing the population based on characteristics known to affect medical need.

1. Active RNs employed in the county per 100,000 adjusted population
2. Students enrolled in RN programs in the county per 100,000 adjusted population
3. Number of short-term general hospitals
4. Number of short-term general hospital beds
5. Ratio of average RN salary to median income
6. Number of nursing and personal care facilities
7. Percent of the population with income below poverty level
8. Population per square mile
9. Ratio of RNs to hospital beds
10. Number of hours per week paid for agency RNs
11. Number of overtime RN hours per week
12. RN vacancy rate
13. RN turnover rate
14. Ratio of LPNs to RNs
15. Total number of budgeted RN positions
16. Percent non-Hispanic white