Evidence Based Research

 

This page consists of four sections

1. Research Outcomes

2. Implementation of Stages 1-3

 3. What do effect sizes and p-values mean

4. A note from the lead author

Research Outcomes


An evaluation of Stages 1- 3 of the programme was the conducted from January 1999-May 2001.  Schools were randomly assigned to  control or experimental groups based on criteria to establish baseline equivalence (RCT). Following this a 3 year quasi-experimental longitudinal research design was implemented.


The experimental schools, who were using the CSP Spelling and Language Programme significantly outperformed the control schools. All of the children in the experimental schools increased their standardised spelling score over the period of the research (Years 2- 4, equivalent Years 1-3 in England). The effect size was 1.19 indicating a very large effect and the p-value was p<.0001.  The average standardised spelling score in the experimental schools was 113 at the end of the research, almost one standard deviation above the mean.


 Implementation of the Stages 1-3


PREREQUISITES


1. LETTER NAMES

To achieve outcomes similar to the research it is important that children in Year 1  (4-5 year old children)  are taught letter names before letter sounds.  Research has shown that teaching letter sounds first can lead to difficulties learning to spell for some children.   There are only 26 letters in the alphabet and 44 sounds (phonemes). Letter sound relationships are not one-to-one and many of these phonemes can be spelled a number of different ways. Children must be able to spell using letter names supported by sounds. It is important that letter names are secure.


2. A SYSTEMATIC PHONICS PROGRAMME SHOULD BE IN PLACE IN YEARS 1 AND 2  (4-6 YEAR OLDS)


It is important to note that the systematic phonics programme used in the research schools prior to the start of the research was 'Letterland'.


It is interesting to note that Ehri (2020) cites 'Letterland' as an example of systematic phonics teaching which is engaging for children.

 On page S57 she states that:

‘Systematic phonics instruction has been mischaracterized as only skill and drill, with little attention to meaning. This is false. Phonics programs may use engaging games or interesting materials to teach letter–sound associations, for example, letter shape–sound picture mnemonics such as Sammy Snake in the Letterland program that Lyn Wendon created (see https://us.letterland.com/). Students apply their letter–sound knowledge to decode words in meaningful texts from the outset’. 


It is important to note that Ehri (2020) does not limit phonics teaching to phoneme-to-grapheme correspondence only. In her 2020 paper she discusses the research she has conducted over the years which has demonstrated the importance of teaching phoneme-to-grapheme correspondences, and on page S56 she points out that:

 ‘Not only beginning reading but also more advanced reading benefits from systematic phonics instruction focused on teaching multiletter units to decode words. To move into the consolidated alphabetic phase, students need to be taught spellings units that include onset-rimes, syllables, and morphemes. Knowledge of these units enables students to decode unfamiliar multisyllabic words and to store these words in memory for sight word reading and spelling.’


Ehri, L.C. (2020) The Science of Learning to Read Words: A Case for Systematic Phonics Instruction. Reading Research Quarterly, 55 (S1) pp. S45–S60 | doi:10.1002/rrq.334


Ehri (2024) also advises that:


'Graphemes are one or more letters representing individual phonemes, the smallest units of sound in a language. Spelling units used to form connections include graphemes representing phonemes (e.g., B-/b/, S-/s/, WH-/w/), onset-rime units (e.g., str-ing, bl-ast), multi-letter units representing syllables (e.g., fan-tas-tic), and morphemes (e.g., up-hold-ing).' (page 4)


Ehri, L.C. (2024) Clarifying the Role of Orthographic Mapping in Sight Word Reading. The Reading League Journal. pp. 4-12


IMPLEMENTATION



The daily testing in Stages 1-3 as outlined in the manuals (the opt in to do more approach) is essential to establish the orthographic processing pathways needed to establish the spellings in memory. TESTING ONCE A WEEK IS NOT SUFFICIENT


The workbook activities should not be given to children to do without the whole class teaching to develop vocabulary knowledge,  word meanings (morphemic knowledge) and usage.  The workbook activities are a follow-up activity to whole class teaching.


There are implementation videos to help schools implement the programme. In addition to these videos schools who purchase the programme can have an online meeting with the lead author. This is usually two to three months after implementing the programme to provide answers for any questions schools may have, and for any further clarification needed.

What do effect sizes and p-values mean


Effect sizes show the magnitude of the difference between groups. The effect size reflects the strength of the intervention’s impact. It shows how meaningful or substantial the effect is in practical terms.


0.2 small effect

0.5 medium effect

0.8 large effect

1.0+ very large effect



P-values provide a measure of the extent to which the result is due to chance.


p < .05  Less than a 5% chance the result is random.  Statistically significant -  Moderate evidence


p < .01  Less than a 1% chance the result is random,  Highly significant-  Strong evidence


p < .001 Less than a 0.1% chance the result is random.  Very highly significant -Very strong evidence


p < .0001 Less than a 0.01% chance the result is random.  Extremely significant- Extremely strong evidence



A note from the lead author


An evaluation of Stage 1- 3 of this programme was the subject of my Ph.D research.  This research met the criteria for rigorous research as outlined below.


  1. Schools were randomly assigned to the control or experimental groups based on criteria to establish baseline equivalence (RCT). Following this a 3 year quasi-experimental longitudinal research design was implemented
  2. Information on participants was provided for study replication 
  3. Standardised scores were used  (reading ages and spelling age i.e., age equivalent scores would not meet the requirement for rigour)
  4. Fidelity of implementation was very closely monitored
  5. The research assistants marking and cross marking the standardised tests did not know whether the tests papers they were marking were from control or experimental schools ( a requirement for blind evaluation)
  6. .The assessment of spelling in independent writing samples, and the evaluation of the quality of independent writing samples, was completed by experienced teachers and inter-rater reliability was established. These teachers did not know whether the samples of writing were from the control or experimental schools (a requirement for blind evaluation)
  7. To ensure the validity of the data, and protect me as the lead author of the intervention, this was a rigorously controlled study. I did not administer the tests, collect any of the data, mark any of the standardized tests or evaluate the samples of writing. This was to ensure the research met the criteria for blind evaluation
  8. The data was entered into SPSS by staff in the School of Education at the lead author’s university, to safeguard its integrity