Learning Systems Design and Development Competencies
Competency: Conducting Needs Assessment
Conducting needs assessment and evaluating learning systems
- Needs Assessment Report (pdf)
This is a summation of a needs assessment review done on an University academic department's end of semester faculty evaluation process. The review was part of the requirements for the Needs Assessment for Instructional Design course, ISLT 9474. I was responsible for reviewing the evaluation process to see if that appropriate steps have been put in place to ensure confidentially of student evaluations and that all students are given the opportunity to participate. Data gathering methods included interviews, observations and surveys of office staff, students and faculty along with review of current articles on evaluations. The report includes descriptions of the data gathering process, as well as examples of the interview protocol, survey and observation instruments.
As I worked on this project, I thought about the differences between doing an assessment for a group that I have to work with everyday with doing an assessment for a group that I'd not worked with before. I realized that the interviewees, etc, were comfortable with my presence and I knew more about the workplace history. So being able to blend in during observations was much easier. I wasn't some outsider hired by the "boss." Working on the assignment, I recognized that sometimes these needs assessments can be viewed as "witch hunts" by some. I tried to make sure that the data gathering methods were as neutral as possible and that the results would represent the truth, without worrying about offending individuals.
Even though the recommendation was for no major changes, I felt that the instruments I created to support the findings covered a lot of information. The interview protocol in combination with the observations and surveys provided a lot of information and was well summarized in the report. On the other hand, I felt that research into options for online evaluations from both articles and experts could have added a bit more to the findings. Time should have been set aside to talk with Academic Resource Center, ARC, to see if they provided options for online evaluations, especially for those students that are participating in the off-campus programs. Looking back at this report, I think that an interview with an ARC representative and perhaps with individuals from other departments about their expectations for the evaluation process might have added more to the report findings.
- Smart Board Adoption (pdf)
As part of my coursework for ISLT 9475, Diffusion of Educational Innovations, we developed a checklist for evaluating the adaptation of an innovation. This activity report is my preliminary summary of observations, a focus group and interview of the adoption and use of a Smart Board by instructors. To me developing the checklist helped summarize different levels of usage, from Optimal to unacceptable, and could be used to illustrate expectations without placing fault or blame. This is a summary only, there are no major conclusions, but it could be used as part of an evaluation or a needs assessment. The checklist documents the expectations at the time it was created. I think more time could be spent on improving the levels of optimal, acceptable and unacceptable usage. In addition to improving, I'm sure there are more usage components that could be added to the list. In addition, the list, as well as the focus groups and interviews, should be used as guidelines as to what additional trainings are either needed or requested.
Competency: Designing Learning Environments
Designing learning environments, including systems for direct instruction, constructivist learning, collaborative work, and performance support
- Direct Instruction Prototype (pdf)
This is a prototype of two direct instruction lessons for ultimate. Using the design techniques like backward design and cognitive model theory for creating instructional materials, I figured out the goals and understandings I wanted to cover and start to lay out the framework of the design. As I was creating the lessons, I tried to keep in mind what I know of the game and what it was like when I learned the game. I wanted something that would keep the players interested and involved. While working on the project, I found that being able to devote larger chucks of time, helped me get immersed in the design and where it's was going. However, I have to remind my self to step back from time to time, so that I don't get too focused on the details. As I fleshed out the outline of the lessons, I keep the class readings near at hand. Looking at articles or fellow student suggestions, for ways to include or combine them into the lessons. Once the lessons were mostly complete, they were used on a small pilot group. After these "players" reviewed the lesson (see the artifact for data and feedback from the pilot users, their responses were taken into consideration, as well as my own responses to the delivery for future revisions of the lessons.
One of the strengths I felt in designing the lesson was that I was very familiar with the subject matter. As a result I was able to come up with a variety of ways to present the topics. Presenting the information in a variety of ways will help the players, better understand and retain the lessons. One of the weaknesses is also one of the things I would have done differently is having a larger group of individuals during the initial tests of the prototypes. Because you need at these 14 individuals to play a game, finding that many new players that are interested in sitting still for the lessons. One possibility would be getting together either with local physical education instructors or kids groups to have a short beginning lesson.
- DI concept map (gif)
I created this concept map as a final project for my designing direct instruction. It was a final project to summarize and illustrate the class learning. Though it's not a whole learning environment, it can be used as a learning aid. I shared it with a friend in an education program at another program and she said it was a great way to visualize how it all fits together.
Competency: Developing Learning Systems
Developing learning systems applications or components of applications
- Web App Dev II Homepage (html)
My first assignment for the Web Applications Development II class was to developing an assignment page for posting assignments and practices. Other individual assignments during the course include creating a guestbook, creating help files and inc files, there is more description of those individual projects on the page. That first project assignment required that I provide links to the required pieces for each individual project assignment and indicate if it was complete. It included an extra option and a recommendation to create administrative functions, but the admin side was not required. After deciding on a possible layout of the assignments page, I created the admin side with login "user" / "enter" so that I could just add and edit assignments and practices as the course progressed.
I enjoyed working on this project. One of the strengths is that it is not just another listing of projects, as I used an image of a check mark to show if a project was complete, rather than just adding more text. The biggest strength of the page is that I can add and update assignment information without needing to get back into the html code. Because of the time that has passed since I completed the class, I can see were there is documentation of how things work that is missing. If I were still working on this project, I would not see these information gaps. Taking that time away from a project really does help with finding the shortcomings and gaps. After the time away, I can see where programming or information needs updating or expansion to fill those gaps.
- Document Editor Prototype (ppt)
Doc. Ed. User Evaluation (pdf)The Document Editor Prototype and user evaluation were part of my coursework for ISLT 9410, Designing Computer Supported Cooperative Work (CSCW). The assignment was to create an improvement to our class web space that would support cooperative work. Other cssw environments I researched during the class influenced the design of some of the features of the prototype. The idea behind the prototype is a space for groups to collaborate on projects. Group members are able to review each other's work, leaving notes, etc. The evaluation is an example from a small user group that I had test the prototype. The evaluation consisted of a short scenario with a user walkthrough followed by a survey. Having this preliminary prototype interactive, instead of a static drawing, helped the reviewer understand a little bit more about how it could work. The idea of adding a cooperative work environment, where the users can visually see when people have logged into the system and where they are in the space, helps increase the group cohesiveness. It can help make the distributed nature on line interactions closer to a traditional in person classroom experience. To me, one of the weaknesses of the project, is that it's not fully explain. It needs to go through more revisions to be a better representation of the idea. When I look at the prototype, a year later, some of the features still make sense and others need further explained or refined.
- Online Testing Proposal (pdf)
This is a group proposal for the ET@MO Series on Academic Transformation submitted in end of 2005 by the MU School of Social Work. The proposal was for redesign of several tests for the Master's of Social Work, MSW, students. The background for the tests is that they used as test out exams for four core classes by entering MSW students. In order to take the tests, students would have to come to the department and take a paper-based test in order to test out of a course. In addition, students only had the opportunity to take the tests once before the semester started and the turn around for scores was generally several days later. The main goal of the project proposal was creation of a pre and posttest instrument that would help the school be able to better assess the understandings of knowledge and skills that the MSW students had acquired upon graduation. In addition to assessing student understandings, the tests would be available online, with results at completion, for two to three weeks before students began classes. This additional time would allow students more flexibility in completing the tests.
The senior faculty member on the team organized and wrote the proposal. My initial role in the project proposal was helping with goal development for the system. After the ET@MO Series on Academic Transformation accepted our grant proposal, my role was to design, administrate and maintain the test site. The four tests have a test bank of about 500-700 questions, from which 100 are pulled randomly for each student. After looking at several options, the group chose to use blackboard to host the test. Faculty members collaborated to create the four tests banks and then sent the banks to me to upload into the new space. The system was put into use for the students that started fall 2006 semester. Strengths of the process and the system itself are that it simplifies the test process for faculty and allows students the ability to access the tests on when and where they want. Being able to access the tests online is also one of the weaknesses of the tests. On average, the students have a good grasp of using online applications like Blackboard, however, there are always a handful students that are either not comfortable with the internet or experience difficultly with learning how to use the system. After the beginning of the summer semester, the group is going to look at the results from the pretests and decide if there are needed changes for the next group of tests.