Projects Underway

Project Description
Throughout SUNY and our national educational system there is a growing need to understand, and therefore support and increase student success. (NYSUNY 2020) Plenty of research has gone into defining characteristics for student success though no magic bullet has been found. The authors of this project assert that continued assessment and use of assessment data to create interventions and redesign curriculum lead to a state of continued improvement for the institution, its faculty and students while maintaining a strong relationship to the current tools used in any of the disciplines.

Preliminary Research
Current researchers worked in partnership from 2007-2010 fulfilling a request to focus on the potential readiness for online learning of teacher aides in New York City. At the onset of this project it was determined more valuable to research and develop an instrument that would assess the readiness of any online learner. After a 3-stage validation process the project culminated in 2010 with a validated survey named: Online Learner Readiness Survey, Dray & Miszkiewicz (2010) and a published article Dray, Lowenthal, Miszkiewicz, Ruiz-Primo Aracelli and Marczynski (2011) about their process. Researchers proposed two main survey constructs (Learner Characteristics & ICT Engagement), though factor analysis has subsequently revealed five underlying constructs: Learner Characteristics, Access to ICT, Use of ICT, Skill with ICT and Relationship to ICT, Dray, et al. (2011). A scoring rubric was developed and disseminated as well, Marczynski & Miszkiewicz (2011).
During dissemination and adoption of the initial survey a reconfigured research team from Buffalo State and the University at Buffalo reviewed feedback from active and potential users. This work unveiled a need to better define sub-constructs within the overarching construct of Learner Characteristics. Dabbagh (2007) and DeTure (2004) influenced the researchers to broaden the survey and consider learner readiness as a whole, based on their positions that learner characteristics required for success in online learners are not different from those required for success in traditional learners. Likewise, the need for ICT engagement across the disciplines is not limited to interaction with a learning management system, rather students work with many varied ICTs across disciplines whether online or in the traditional classroom.
Researchers Miszkiewicz, Marczynski (Buffalo State), and Kroll (University at Buffalo) will lead the transition to create and validate the new instrument (SUNY Prep, Learner Preparedness Survey) using samples from University at Buffalo and George Mason University. Researchers are also in contact with the Dean of Empire State College Center for Distance Learning to gain support. Validation using item analysis based on the methods of Converse and Presser (1986) and factor analysis employing principles from McDowell (2006) will be used to determine the validity of the question meanings and the constructs themselves.
Survey questions will be reviewed and rewritten around four new sub-constructs: academic self-concept, academic self-efficacy, learner locus of control, and learner values about outcomes Baran, Gokalp, Maskan, (2011); Torkzadeh, Koufteros, & Pflughoeft (2003); Bong and Skaalvic (2003); Marsh, Byrne, & Shavelson (1988); Schunk (1991), and those previously defined under ICT Engagement: Access to ICT, Use of ICT, Skill with ICT and Relationship to ICT.

Goal
Our vision is to create a system for assessment of learner preparedness with prescriptive interventions that, when implemented, increase individual educational success.
Our goals for Phase One of this project are to develop, and validate an instrument designed to predict student success based on individual learner characteristics and the learner’s level of engagement with information and communication technology (ICT). Phase Two will test the instrument’s ability to predict student success and include the development of interventions to achieve successful learning outcomes. Used appropriately, the instrument can provide information to improve instruction and student engagement in learning across disciplines and across the education pipeline.
The survey as a whole will be rewritten and reorganized with a new scoring system. ICT engagement questions will be updated & rewritten to match the current zeitgeist and facilitate reporting. Validation using item analysis based on the methods of Converse and Presser (1986) and factor analysis employing principles from McDowell (2006) will be used to determine the validity of the question meanings, and the constructs themselves. The resulting new survey will thus be comprised of the four new constructs under learner characteristics: academic self-concept, academic self-efficacy, learner locus of control, and learner values about outcomes and the four existing constructs under ICT Engagement: access to ICT, use of ICT, skill with ICT and relationship to ICT.

Outcomes
The outcome of Phase One is a validated instrument and scoring rubric to assess learner readiness (SUNY Prep, Learner Preparedness Survey). Educators will have data in the form of individual student reports for use by academic advisors, faculty and students themselves as well as comprehensive group reports (i.e., single class, an entire program) designed to assist program directors, curriculum designers, etc. with program revisions and prescriptive intervention strategies to increase the overall probability of student success.
Phase 1, Tier 2 funding is being sought to support the initial development of this innovative measure of ICT skills and related learner characteristics to match the substantial in-kind investment of the development team. The next step will be a Phase 2 project request to examine predictive validity and to develop specific interventions based on cutoff scores of the instrument.

Project outcomes provide a foundation for a new instrument to assess learner characteristics and success in online learning.