Improving the Training Resource Review Process

Originally published at eLearningIndustry.com on 10 October 2013.

When you send training resources out for review, are you getting back the type of constructive feedback you need? This article outlines 3 levels of review, and gives you 6 tips on how to apply these levels to get great feedback and take back control of the training resources review process.

Arranging a formal review of training resources can be a frustrating process. I tend to get a lot of comments about spelling and grammar, or about the colors and imagery I have used. But I struggle to get meaningful feedback on the accuracy and quality of the information within my resources.

This isn’t surprising. Most people are never formally taught how to review documents. It’s a skill you pick up as you go along. And there are different types of review – ranging from simple proofreading to complex subject matter analysis. I believe very few people understand that there are different types of review, and never develop skills at the more complex review types.

I have started implementing a three-step review process for training resources. I run each review in sequence, and ask different people to complete each type of review. When I send resources for review, I always give specific information on what aspects of the resources I want feedback on. Here are my three review levels, with the areas I ask people to review in each.

3 Review Levels To Improve The Training Resource Review Process

1. Quality check

Choose reviewers with good administrative and language skills. Personal Assistants or Team Administrators are generally good quality checkers.

  • Are there any spelling, grammar or punctuation mistakes?
  • Are there any technical issues (bugs)?
  • Is the language simple and easy to understand? Has Plain English been used?

2. Design analysis

Choose reviewers with good understanding of instructional design or web design principles. Learning & Development specialists are best.

  • Does the structure of the resource make sense? Do the topics flow logically from one to the next?
  • Is the resource easy to navigate? Can you quickly find information on a specific sub-topic?
  • Is the look & feel of the resource appropriate (e.g. the appropriate colors, fonts & images for the subject matter)?
  • Is the look & feel cohesive (e.g. do the colors, fonts & pictures work well together)?
  • Will the resources meet the learning objectives of the project?
  • Are the resources engaging? Will learners be bored or overwhelmed?

3. Subject matter review

Choose reviewers who understand the content of the training. These are usually representatives of a business unit or a project team.

  • Is the information presented accurate?
  • Have all the relevant points on this topic been discussed? Is any information missing?
  • Are the most important points given the most attention?
  • Are complex and confusing topics given enough attention?
  • Have appropriate scenarios or activities been included? Do they relate to what the learners do in their day-to-day work?

6 Tips To Improve Your Training Review

  1. Run your reviews with a group in a meeting room. You can ensure the review happens in your timeframe, and focus your reviewers on the specific aspects you would like reviewed.

  2. Document all feedback you receive. Capture the name of the person that gave the feedback, where in the resource is relates to (e.g. a page or slide number), and what change they would like made. Add a section where you can track whether you have made the change, and provide explanation if you have not.

  3. Remember that you still own the resource. You do not need to incorporate every piece of feedback you receive. But you should provide a valid justification if you do not incorporate a specific piece of feedback.

  4. Thank your reviewers for their time. Make sure they understand that you appreciate their input.

  5. Set end dates for each review round, and fix the number of reviews that can take place. You may not always be able to stick to these dates, but they do help to focus your reviewers and yourself.

  6. Have an end date. No resource will ever be perfect and please everyone. You must be willing to reach a point of ‘good enough’ and start using what you have developed. You can always do another review in six months and make more changes then.

Sample: Introducing eLearning

This is a small module I put together recently in Adobe Flash 5.5 and Adobe Captivate 5.

I wanted to develop a module that was completely non-linear. The topics can be explored in any order. I also wanted to try a module where I didn’t include explicit instructions on how to navigate. People must work out how to interact with the module for themselves, which I am hoping will appeal to their natural curiosity and engage them more with the content.

I have disabled some of the links in this sample, but you should still be able to get the idea.

What do you think? Does the lack of clear navigation and logical order help or hinder your learning?

Sometimes it’s about having somewhere to ask the questions

My team was recently discussing what new courses we might offer in our Learning Management System. We didn’t want to just offer training for the sake of it – we wanted training that was useful.

We brainstormed some ideas for topics based on changes that have happened recently in the organisation. And inevitably someone raised the issue that developing training would be very difficult, because the topics are so new that there’s just no content yet – and nothing to start building it from.

Which has led me to my realisation of the day:

Training solutions aren’t always about having all the answers.
Sometimes they are about giving people somewhere to ask the questions.

The social networking tools that we have in our LMS; such as forums, chats and polls, give us a unique ability to support our staff through changes in their business practice over what they can get anywhere else. We give them somewhere that they can share their thoughts and feelings about the change. They can tell us what problems they are struggling with – and THEN we can start developing resources to help them with those problems. But without that initial way of gathering information, supporting them is much harder; and our response is much less timely.

I think training in the future should be geared more to this approach:

  1. Set up a space to discuss the topic.
  2. Analyse the information your audience shares on the topic.
  3. Develop resources targeted to those particular concerns.

This approach also gives your audience the chance to develop and share their own expertise, and makes it easy for people to identify the subject matter experts in their area.

Social media isn’t a new concept, but I think we are still not using it to its full potential. Let’s start building spaces to ask questions. It’s a great idea, right?

Sample: Making WH&S training interesting with storytelling

I was recently looking at enhancing our Workplace Health and Safety (WH&S) training. As with all compliance based training, this is a challenge because the training audience generally doesn’t see the value in the training, and there is a strong temptation to just skim over the content.

My recommended tactics to make the training more effective were:

  1. Remind people early on why Workplace Health & Safety is important.
  2. Present the organisation-specific information first
  3. Give people a quiz to test their knowledge If they really do know it all already, there’s no need to complete the remaining training. But if there are still gaps, they do need to keep going.
  4. Include information specific to the organisation. What are the most commonly reported incidents in our workplace? What are some specific tips for the type of work we do


Unfortunately we didn’t get to implement my proposed plan for various reasons. But while I was working on this I mocked up an introductory video that I’m quite proud of.

This is one of my first attempts at developing solely in Adobe Flash CS5 (rather than using Adobe Captivate 5), and I was very happy with how I could use the same-old generic images from Presenter Media, but make them more valuable by adding simple facial expressions. This may not be as polished as a professionally developed module would be – but I think it is still fairly effective.

Did it work? Evaluating the effectiveness of online training solutions

In March 2013 I presented at the iDesignX Conference in Melbourne. My presentation was about how to evaluate the effectiveness of online training solutions.

Click here to view the presentation.

The key points of the presentation are:

  • Good evaluation begins with good analysis. Ask all the key stakeholders “What are we trying to acheive?”
  • There are five key stakeholder groups: drivers, business owners, project managers, designers & developers, and clients.
  • Incorporate evaluation methods to address the needs of each stakeholder group.
  • If you identify an area where the training wasn’t effective, look for ways to address that both for the current training solution, and for future training solutions.
  • Summarise your evaluation findings in a Training Evaluation Report and send it to all your key stakeholders.

Why I officially hate WordPress

I love Moodle. Do you know why? Because it does what I tell it to do. Specifically, if I press the Enter key, I get a hard return. Or if I decide to add HTML code and I type a < br > break, I get one line break – just one.

But WordPress isn’t like that. Sometimes when I press Enter I get two hard returns. Sometimes I get none. If I use the Visual editor and type a < br > break I might get one line break, or maybe two. It’s finicky. And it’s not consistent. I might preview my page, carefully fix all my line spacing, and then when I hit save a line break somewhere else disappears FOR NO REASON.

I’ve resorted to turning off the Visual Editor and just typing HTML code. Which is a real pain.

I can see from internet searches that this is a wide ranging problem, and has been for some time. Why, oh why hasn’t it been fixed?

Not that I’m being proactive and volunteering to help or anything. I just like to complain.

OK, end of rant.

Nifty way to prepare dates for MS Excel PivotTables

I often create PivotTables in MS Excel. I have had a recurring hassle when I use dates and want to analyse data by month. If I simply format my column to show only month and year, the pivot table will not group my information together.

Formatted date in MS Excel

The custom formatting displays month and year, but the Format bar shows a full date and time.

Pivot Table 1

I end up with three instances of July 2013 – one for each different date and time

Historically I would create a new column, type in the month, and then copy and paste for each row in that month. But I have now found a much better way to work around this problem:

  1. Select the column heading for the date column (e.g. column C).
  2. In the menu bar select Format > Cells. Select the Custom category. Enter your preferred format in the Type field (e.g. mmm yy). Click OK.
  3. Press the Ctrl + A keys on the keyboard to select all text. Press Ctrl + C to copy.
  4. Open a blank MS Word document. Press the Ctrl + V keys to paste the text. MS Word will represent the dates with your preferred formatting.
  5. Press the Ctrl + A keys again, then Ctrl + C keys again to select and copy the text.
  6. Return to MS Excel. Press Ctrl + V to paste.
  7. Create your PivotTable. The dates will now display in the correct format.


MS Excel dates in correct format

After copying through MS Word, the Formula bar now shows 01/07/2013 for all July cells.

Pivot Table 2

The PivotTable now shows one row for each month.

Stop ringing your bells and blowing your whistles

In my last role change I noticed a change in the way my colleagues approach training solution design and development. My last role was very focussed on developing solutions in a cost and time effective way. Sometimes quality had to be sacrificed to get the solution in on time. In my current role I have noticed people are willing to take more time to get things right. If the deadline is coming up, and the quality isn’t right, well the deadline just needs to slip.

I really don’t like this second way of thinking. It grates on me. And here is why:

1. Time is money

Let’s say we have two developers who are creating an eLearning module to explain some changes to a process. Each charges out their time at $600 per day.

  • One developer creates a simple module with stock images of people, audio voiceover and a simple quiz. It takes them a day. Therefore their time is worth $600.
  • The second developer creates an interactive module with video, interactive drag and drop activities and Flash animations. It takes them three days. Therefore their time is worth $1,800.

  • Is the second module three times as good as the first one? Are people getting three times as much value from it?
    I would argue the answer is ‘no’.

    In my experience motivated learners will understand the content regardless of how it is presented (provided the information is clear).

    The times when the extra bells and whistles are most useful is when you have unmotivated learners. When someone doesn’t WANT to engage with the content, making it more visually, auditorially and kinaesthetically appealing can help. But otherwise it’s just not necessary.

    2. A less than perfect solution is better than no solution at all

    Let’s say we have two developers who are creating a complex eLearning course on the new performance management policies and procedures.

    • One developer takes the simple route and finishes development in a month.
    • The second developer decides to get a perfect solution, and finishes development in four months.


    Where are the learners getting their information during the time the training is being developed?.

    If you have information that is time-critical, it is important to get the information out fast. If your audience learns a bad behaviour because they don’t know any better and don’t have the right information, it will be doubly difficult to change that behaviour. And when your perfect training solution finally rolls out, some of them may be thinking “why do I have to do this training – I know how to do this already”.

    If the information is not time-sensitive (e.g. soft skills training), taking some more time to do something impressive might be worth it. Maybe.

    The moral of the story

    When you do your Training Needs Analysis, always consider a) how time-critical the information is, b) what the intrinsic value of the information is, and c) how motivated your audience is to learn. If it’s important, valuable information that people need now, just get in there and write it. Don’t worry about making it look pretty.

Mystery solved: Runtime borders on Adobe Captivate rollover slidelets

For years now I have been aggravated when using rollover slidelets in Adobe Captivate because they kept adding these ugly blue borders, even if I turned off the runtime borders.

An example of the dreaded runtime border

I have finally worked out what I was doing wrong.  There are TWO Show Runtime Border checkboxes.

  • One for the rollover area, under the Transition heading in the Properties panel.
  • One for the slidelet, under the Fill & Stroke heading in the Properties panel.

 

If you deselect both checkboxes, the problem goes away.

The checkbox in the Transition panel  The checkbox in the Fill & Stroke panel