This week is also another meeting of the UW System Board of Regents. Consistent with yesterday's blog, here I am sharing some thoughts from one of my students who watched the February 9 meeting of that Board. This was the meeting at which members discussed rising costs, cost containment, and the potential for cutting enrollment throughout UW System.
"The debate following the panel of chancellors reflected the three corners of the Iron Triangle: access, funding, and quality. Surprisingly, most of the concerns seemed to revolve around issues of quality, and to some extent access, not around funding concerns...When pressed on the issue, [President Kevin] Reilly was forced to admit that he did not know when or if quality decline would result in reduced enrollment...
Chancellors Sorenson and Wachter noted students' reluctance to leave universities for the workforce. While not using the same intense and accusatory rhetoric of a Jackson Toby, they did claim that students lack efficiency in pursuing their education. At this point [Vice-President] Mark Nook interjected to provide an anecdote about how his daughter ... had managed to graduate in 4 years...This reflected a misguided assumption that his daughter's experience is typical, rather than the reality that 3/4 of today's college students face serious constraints and pressures that could impede their academic progress.
In one particularly poignant moment, Regent Jose Vasquez questioned how the System would provide for students of color and those with disabilities. He was the only member to directly address issues of access for underserved populations. He noted that these students cost more than what he called 'ideal' and 'easy' students, and wondered how they'd be impacted by cost-cutting measures. His remarks highlighted the the non-financial values of public higher education and provided a moment to undercut finance as the "privileged language of reality." Sadly, none of the board members or chancellors responded to his concern."
Thursday, April 12, 2012
Wednesday, April 11, 2012
Students Respond to the UW Taskforce
Today the Wisconsin Legislature's "Special Task Force on UW Restructuring and Operational Flexibilities" meets again in Madison. In honor of that, I want to bring you some student perspectives on one of the prior task force meetings--the one that took place on February 8, and included presentations from the chancellors of UW-Madison and Milwaukee. I'm doing this because student voices are notably absent from these meetings-- students have not been given a chance to present (they will, for the first time, on May 9) and they do not serve on the task force. A few have written letters or spoken publicly on the topic, but most have not.
Recently, students in my Introduction to Debates in Higher Education Policy course (EPS 518) were asked to view a legislative or regents hearing or meeting of their choosing and write a response paper. Below, I provide some representative examples of their responses -- these are deliberately provided without attribution to the student (all are undergraduates) and are posted with their specific permission. My intention is to simply allow the voices of students to emerge, as I think their comments and questions are critical to the discussion. If other students wish to share their considered opinions of hearings, please do send me your memos, and I'm happy to post thoughtful excerpts.
Student 1: "..The nature of the meeting itself...was self-congratulatory and generally insufficient in data. The meeting focused on individual knowledge and individual power, that is they spoke of their personal bailiwicks, which, while it makes sense for a panel of experts, was insufficient...though the panel brought up several reforms, these reforms were often self-serving, under-supported by data, and/or uncertain in their impact."
Student 2: [Flexibilities were a primary topic of discussion at the hearing and yet] "there was an utter lack of understanding about what was being discussed...Despite the apparent knowledge gap about what flexibilities were, they were the main focus of discussion and seemed to be the only thing anyone believed could save the UW System money...What is perplexing about the deregulation rhetoric is that, according to Gary Rhoades, this behavior is...a trickle-down model of funding. In exchange for deregulation and flexibilities, institutions receive less state support. This ends up privileging the elite institutions while creating problems for local institutions. However, it was chancellors of schools like UW-Oshkosh and Platteville who were calling for this deregulation..I cannot help but wonder why the chancellors of these schools would call for deregulation when it would mean less money from the state."
Student 3: "I was surprised at the small number of women on the task force-- just 3. I was disappointed at the lack of minority representation, but not surprised....[many spoke about the word 'product'] and the word 'product' is a difficult one, and its use underscores the different positions and value systems of the task force members. [Most] seemed to think that having a better education and a lower price were mutually exclusive things, and that one must be sacrificed for the other."
Student 4: "As a student, a major concern became evident at this meeting. Members of this task force have been charged with creating innovative solutions to the challenges facing the UW System, challenges that have arisen from a lack of funding. The majority of task force members, however, are not even close to specialists in higher education, let alone public higher education. In fact these people who are supposed to be coming up with solutions are primarily business people who have spent most of their professional careers in the private sector. [Thus] it is clear these are powerful voices denouncing the importance of public funding for various reasons."
Recently, students in my Introduction to Debates in Higher Education Policy course (EPS 518) were asked to view a legislative or regents hearing or meeting of their choosing and write a response paper. Below, I provide some representative examples of their responses -- these are deliberately provided without attribution to the student (all are undergraduates) and are posted with their specific permission. My intention is to simply allow the voices of students to emerge, as I think their comments and questions are critical to the discussion. If other students wish to share their considered opinions of hearings, please do send me your memos, and I'm happy to post thoughtful excerpts.
Student 1: "..The nature of the meeting itself...was self-congratulatory and generally insufficient in data. The meeting focused on individual knowledge and individual power, that is they spoke of their personal bailiwicks, which, while it makes sense for a panel of experts, was insufficient...though the panel brought up several reforms, these reforms were often self-serving, under-supported by data, and/or uncertain in their impact."
Student 2: [Flexibilities were a primary topic of discussion at the hearing and yet] "there was an utter lack of understanding about what was being discussed...Despite the apparent knowledge gap about what flexibilities were, they were the main focus of discussion and seemed to be the only thing anyone believed could save the UW System money...What is perplexing about the deregulation rhetoric is that, according to Gary Rhoades, this behavior is...a trickle-down model of funding. In exchange for deregulation and flexibilities, institutions receive less state support. This ends up privileging the elite institutions while creating problems for local institutions. However, it was chancellors of schools like UW-Oshkosh and Platteville who were calling for this deregulation..I cannot help but wonder why the chancellors of these schools would call for deregulation when it would mean less money from the state."
Student 3: "I was surprised at the small number of women on the task force-- just 3. I was disappointed at the lack of minority representation, but not surprised....[many spoke about the word 'product'] and the word 'product' is a difficult one, and its use underscores the different positions and value systems of the task force members. [Most] seemed to think that having a better education and a lower price were mutually exclusive things, and that one must be sacrificed for the other."
Student 4: "As a student, a major concern became evident at this meeting. Members of this task force have been charged with creating innovative solutions to the challenges facing the UW System, challenges that have arisen from a lack of funding. The majority of task force members, however, are not even close to specialists in higher education, let alone public higher education. In fact these people who are supposed to be coming up with solutions are primarily business people who have spent most of their professional careers in the private sector. [Thus] it is clear these are powerful voices denouncing the importance of public funding for various reasons."
Monday, April 9, 2012
The Value of Looking at Local Results
The report we released today has an interesting history that shows the value of looking beyond the initial results of an experiment. Later this week, we are presenting a paper at AERA entitled "In School Settings, Are All RCTs Exploratory?" The findings we report from our experiment with an iPad application were part of the inspiration for this. If Riverside Unified had not looked at its own data, we would not, in the normal course of data analysis, have broken the results out by individual districts, and our conclusion would have been that there was no discernible impact of the app. We can cite many other cases where looking at subgroups leads us to conclusions different from the conclusion based on the result averaged across the whole sample. Our report on AMSTI is another case we will cite in our AERA paper.
We agree with the Institute of Education Sciences (IES) in taking a disciplined approach in requiring that researchers "call their shots" by naming the small number of outcomes considered most important in any experiment. All other questions are fine to look at but fall into the category of exploratory work. What we want to guard against, however, is the implication that answers to primary questions, which often are concerned with average impacts for the study sample as a whole, must apply to various subgroups within the sample, and therefore can be broadly generalized by practitioners, developers, and policy makers.
If we find an average impact but in exploratory analysis discover plausible, policy-relevant, and statistically strong differential effects for subgroups, then some doubt about completeness may be cast on the value of the confirmatory finding. We may not be certain of a moderator effect--for example--but once it comes to light, the value of the average impact can also be considered incomplete or misleading for practical purposes. If it is necessary to conduct an additional experiment to verify a differential subgroup impact, the same experiment may verify that the average impact is not what practitioners, developers, and policy makers should be concerned with.
In our paper at AERA, we are proposing that any result from a school-based experiment should be treated as provisional by practitioners, developers, and policy-makers. The results of RCTs can be very useful, but the challenges of generalizability of the results from even the most stringently designed experiment mean that the results should be considered the basis for a hypothesis that the intervention may work under similar conditions. For a developer considering how to improve an intervention, the specific conditions under which it appeared to work or not work is the critical information to have. For a school system decision maker, the most useful pieces of information are insight into subpopulations that appear to benefit and conditions that are favorable for implementation. For those concerned with educational policy, it is often the case that conditions and interventions change and develop more rapidly than research studies can be conducted. Using available evidence may mean digging through studies that have confirmatory results in contexts similar or different from their own and examining exploratory analyses that provide useful hints as to the most productive steps to take next. The practitioner in this case is in a similar position to the researcher considering the design of the next experiment. The practitioner also has to come to a hypothesis about how things work as the basis for action.
-- DN & AJ
We agree with the Institute of Education Sciences (IES) in taking a disciplined approach in requiring that researchers "call their shots" by naming the small number of outcomes considered most important in any experiment. All other questions are fine to look at but fall into the category of exploratory work. What we want to guard against, however, is the implication that answers to primary questions, which often are concerned with average impacts for the study sample as a whole, must apply to various subgroups within the sample, and therefore can be broadly generalized by practitioners, developers, and policy makers.
If we find an average impact but in exploratory analysis discover plausible, policy-relevant, and statistically strong differential effects for subgroups, then some doubt about completeness may be cast on the value of the confirmatory finding. We may not be certain of a moderator effect--for example--but once it comes to light, the value of the average impact can also be considered incomplete or misleading for practical purposes. If it is necessary to conduct an additional experiment to verify a differential subgroup impact, the same experiment may verify that the average impact is not what practitioners, developers, and policy makers should be concerned with.
In our paper at AERA, we are proposing that any result from a school-based experiment should be treated as provisional by practitioners, developers, and policy-makers. The results of RCTs can be very useful, but the challenges of generalizability of the results from even the most stringently designed experiment mean that the results should be considered the basis for a hypothesis that the intervention may work under similar conditions. For a developer considering how to improve an intervention, the specific conditions under which it appeared to work or not work is the critical information to have. For a school system decision maker, the most useful pieces of information are insight into subpopulations that appear to benefit and conditions that are favorable for implementation. For those concerned with educational policy, it is often the case that conditions and interventions change and develop more rapidly than research studies can be conducted. Using available evidence may mean digging through studies that have confirmatory results in contexts similar or different from their own and examining exploratory analyses that provide useful hints as to the most productive steps to take next. The practitioner in this case is in a similar position to the researcher considering the design of the next experiment. The practitioner also has to come to a hypothesis about how things work as the basis for action.
-- DN & AJ
Subscribe to:
Posts (Atom)