[CLOSED] Question & Answer session on Qualification Achievement Rates (QAR) 2018 to 2019 with the ESFA

Home Forums Data issues [CLOSED] Question & Answer session on Qualification Achievement Rates (QAR) 2018 to 2019 with the ESFA

Tagged: 

This topic contains 61 replies, has 23 voices, and was last updated by  Laura ESFA 3 weeks, 6 days ago.

Viewing 25 posts - 26 through 50 (of 62 total)
  • Author
    Posts
  • SFA STAFF

    Laura ESFA
    Keymaster

    Question:
    The English and Maths GCSE tab has a breakdown of the grade distribution. This graph appears to be using all starts as the dataset, and not taking account of exclusions (i.e. it includes non-starts and transfers).

    Please can you confirm that this is correct and that it will be corrected for the final release?

    Response from the ESFA:
    The technical team are aware that this page is currently displaying both inclusions and exclusions. A piece of functionality is in development to allow the user to filter on inclusions/exclusions, however we can’t yet confirm when this will be implemented.

    Thanks,

    Laura

     
    #420994
    SFA STAFF

    Laura ESFA
    Keymaster

    Question:
    We have some apprentices where the frameworks are coded with completion:outcome statuses of 2:8 in R14 18/19 and 2:1 in 19/20, where the 19/20 R04 file has not been matched as the latest record for this aim. These are showing in our timely QAR as non-completers. Likewise, we have some that were coded as 1:null in 18/19 and 3:3 in 19/20 that have been matched to the latest file. All the associated matching elements as per the technical specifications are consistent.

    Please can you confirm whether using the record from the 19/20 R04 file is dependent on any criteria (such as the completion status at R14) or whether it should always be used if present in the 19/20 file.

    Response from the ESFA
    Hi there,

    Thank you for your question.

    We use R04 in the overall and timely QAR calculations. The planned or actual end dates must be in 2018/19 to be counted in this year’s QAR. The Timely calculation is looking for an achievement within 90 days of the planned end date (planned end dates up to 31 July 2019) – if the actual end date was more than 90 days after the planned end date it will not be counted as a timely achievement but it would be included as an overall achievement.

    Thanks,

    Laura

     
    • This reply was modified 1 month, 1 week ago by  Laura ESFA. Reason: clarity
    • This reply was modified 1 month, 1 week ago by  Laura ESFA.
    #420996
    SFA STAFF

    Laura ESFA
    Keymaster

    Follow up response to missing QAR dashboard:
    We are a sixth form college with 16-19 students.

    Response from ESFA:
    HI Stephen,

    If you are a sixth form college that has converted to an academy then the absence of a QAR dashboard could be correct. In the first year of conversion to a MAT you will have a QAR dashboard to reflect the ILR submitted before conversion.

    However the year after conversion, you will not have a QAR as you are funded through the schools funding model, not through the ILR via funding model 25. You should still be able to see historical QARs using your new UKPRN, though.

    Thanks,

    Laura

     
    #421000

    lesleyvalentine
    Participant

    We have 2 students who were 15 at the start of their course – they have been included in the QAR but have not been counted as a leaver or achiever although they both got good grades.
    Lesley

     
    #421020

    JBankhead
    Participant

    Please could you explain the Minimum Standards and how they are calculated and why some qualifications are below threshold or direct me to the appropriate explanation in the guidance.

    Thanks

     
    #421042
    SFA STAFF

    Laura ESFA
    Keymaster

    HI all,

    I just wanted to provide some additional clarification as the ESFA has received a number of
    queries from Sixth Form Colleges that have converted to academy status asking why they cannot see a report this year.

    This is because we do not produce 16-18 QAR for school funded 16-18 programmes. If a Sixth Form College continued to send data on Funding Model 25 for the remainder of the year in which they converted then they will have a report for that year. However, if they only reported their data as a school / MAT in the following funding year then they will no longer receive 16-18 QAR reports.

    Thank you again for all your questions and comments. If your question has not yet been answered, please be assured that responses are in progress.

    Thanks,

    Laura

     
    #421136

    Paul Blackshaw
    Participant

    The accuracy and integrity of the QARs seems to be diminishing at an alarming rate.

    I have learner achievements in my R12 QAR correctly counted as 2018/19 achievements but in my R14 QAR they are not. They appear in R12 with a hybrid end year of 2018 but in R14 they are 2018.5. The ILR data for these learners remained unchanged from R12 to R14, so can you please explain why the two QARs have treated their records differently?

    And can you please provide an answer to the excellent question posted by Emma Ford on January 13, 2020 at 11:52am? Waiting until January 2020 to publish a significant change to the methodology for the calculation of 2018/19 overall Qualification Achievement Rates for Apprenticeship Standards on Funding Model 36 is unacceptable and should not be allowed.

     
    #421229
    SFA STAFF

    Laura ESFA
    Keymaster

    Hi Paul – I am just awaiting some additional clarification from the QAR team before posting a response to Emma’s question. Please be assured that all questions will receive a response.

    Thanks,

    Laura

     
    #421247

    kp@nclt
    Participant

    Hi

    We have a sixth form college that converted to academy within the MAT in 17-18 and have always submitted ILR on funding model 25. We don’t have a QAR – why is this?

    We also have a 16-19 free-school that also submits ILR on funding model 25 and this college does get a data file but not the dashboard.

    I am finding the inconsistencies really frustrating.

    Thanks
    Karen

     
    #421275
    SFA STAFF

    Laura ESFA
    Keymaster

    Questions:
    Hi, we have a few questions we would appreciate if you could answer:
    1 – Can we have more clarification on timely achievement impacting QAR?
    2- Have ESFA revised the 62% benchmark after moving to standards?
    3- Can we publish QAR 18/19 for our industry ?
    4- Will QAR rates have an impact on visit cycles ?
    Many Thanks, looking forward to hear from you

    Response from ESFA:
    Hi Livia,

    Thank you for your questions. Answers as follows:

    1. The Overall QAR and Timely QAR are separate measures. Overall uses the Hybrid end year whereas Timely uses the planned end year +90 days – which is the reason why we need R04 to calculate the timely rate. We are not calculating a timely QAR for standards but will produce a separate table in NARTS.

    2. We review the minimum standards thresholds each year and the threshold remains at 62%.

    3. QAR become official statistics when the NARTS tables are published in March 2020 and can be published at that point.

    4. Could you come back to us on what you mean by visit cycles? (I just want to make sure we give you the right info!)

    Thanks,

    Laura

     
    #421285

    cookco
    Participant

    When I select ‘Download Data’ in View Education Data, the data files available are:

    Prov RO4
    In Year R12
    In Year R10

    Shouldn’t R14 be in here??
    I also have the issue of old records in there that are included in the QAR…

     
    #421327

    steveh
    Participant

    The R04 is the R14 one, I agree it’s a confusing name.

    QARs always include three years of data. You can filter on Hybrid End Year = 2018 to find this year’s. If you had continuers or BILs in previous years, they may be counting against you this year if they fail the rules that apply.

     
    #421335

    SAli
    Participant

    Hi,

    When will the 2018-19 learner data be available to download as I have checked and it is still showing the the in year data only for 2018-19 on viewyourdata.education.gov.uk/Download/QAR_1819

    Kind Regards,

    Sali

     
    #421349
    SFA STAFF

    Laura ESFA
    Keymaster

    Question:
    It was mentioned in July 2019 that data for standard completions would be considered up to October 2019. Is this still the case?

    Response from ESFA
    Hi Sam,

    Just to confirm, Steveh is entirely right in directing you to the latest guidance:
    https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/856551/App_standards_transparency_table_delays_FINAL.pdf

    I believe the document was flagged in the 15 Jan edition of Update but might have just missed the cut off for the 8 Jan edition. I do apologise though if this guidance was hard to find – it’s something I can raise with our content designers.

    Thanks,

    Laura

     
    #421376
    SFA STAFF

    Laura ESFA
    Keymaster

    Question:
    The accuracy and integrity of the QARs seems to be diminishing at an alarming rate.

    I have learner achievements in my R12 QAR correctly counted as 2018/19 achievements but in my R14 QAR they are not. They appear in R12 with a hybrid end year of 2018 but in R14 they are 2018.5. The ILR data for these learners remained unchanged from R12 to R14, so can you please explain why the two QARs have treated their records differently?

    And can you please provide an answer to the excellent question posted by Emma Ford on January 13, 2020 at 11:52am? Waiting until January 2020 to publish a significant change to the methodology for the calculation of 2018/19 overall Qualification Achievement Rates for Apprenticeship Standards on Funding Model 36 is unacceptable and should not be allowed.

    Response from the ESFA:

    We use R04 data when calculating provisional and final QAR, data reported in R04 can change the QAR calculation or the hybrid end year the aim was counted in in the in-year QAR. The purpose of the in-year reports is to help providers return accurate ILR data and identify errors that affect QAR before R14 has closed.

    It was necessary to update the Technical Specifications after R04 had been collected – the reason for doing so was to improve the accuracy and integrity of QAR. This change was necessary because of the changes made in the ILR specification for 2019/20 regarding how the achievement of standards on FM36 are reported. We had expected to make the corresponding change in the 2019/20 QAR business rules but during testing we identified that actual end dates in 2018/19 and achievement dates in 2019/20 had been reported in R04. We were not expecting to see this and identified that some providers may have missed paragraphs 319 to 323 of the 2019/20 Provider Support Manual when returning their R04 ILR data. The change was therefore necessary to identify achievement dates from 1 August 2019 onwards and prevent their incorrect inclusion in the 2018 hybrid end year based on the 2018/19 actual end date reported in R04 because that date is no longer telling us when the standard was achieved. The achievements will be counted in QAR next year using the achievement date reported to identify the hybrid end year.

    We try to avoid making late changes to published specifications but during testing we occasionally find issues that need to be clarified or corrected.

     
    #421406

    Emma Ford
    Participant

    Morning Laura,

    Many thanks for your response in relation to my query.

    Although some providers may very well have missed the guidance in paragraphs 319 to 323 of the provider support manual 2019-2020 V1, I do not feel that it was very well explained from the start.

    As with many of the ESFA’s guidance documents (provider support manual, ILR specification, funding rules to name but a few) the wording is very much open to misinterpretation and often contradicts other sections of said document or others when cross referenced, leading to much confusion.

    Also, in example one on page 43 the achievement date illustrated is incorrect.

    We raised this with the service desk back in July of last year who stated: “there is an error in the guidance, we have fed it back to the team responsible for this guidance and it will be corrected in due course”. V2 of this document has yet to be released.

    The service desk also, on numerous occasions gave very conflicting guidance and advice on recording achievement dates each time we raised an issue as well as taking up to four to six weeks in each instance to resolve any queries we had.

    We also posted topics on this forum and found that some members were as equally confused when attempting to record this information on the ILR correctly.

    If the guidance had of been clearer and more accurate information had of been provided by the service desk at the time perhaps a lot of providers could have recorded this information more accurately and in a timely fashion.

    Emma

     
    #421802

    r012943
    Participant

    Hello,

    After inspecting our apps dataset, I can see that we have some learners included within our leaver cohort that I would’ve expected to be excluded by current rules.

    These learners appear to be learners who are coded as restarts (e.g. they had been on planned break previously) but then upon return to programme, have then withdrawn within their funding qualifying period?

    Have the rulings regarding this scenario changed in that they would now be a countable withdrawal as they are restarts?

    Thanks for any help

     
    #421804

    Martin West
    Participant

    The funding qualifying period would apply only from the original start date not the restart date.
    HTH

     
    #421812
    SFA STAFF

    Laura ESFA
    Keymaster

    Question:
    Please could you explain the Minimum Standards and how they are calculated and why some qualifications are below threshold or direct me to the appropriate explanation in the guidance.

    Response from the ESFA:

    Hi there,

    Thank you for your question.

    Information on Minimum Standards and how they are calculated can be found in the following document: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/810050/Minimum_Standards_201819_v1.pdf

    Thanks,

    Laura

     
    • This reply was modified 1 month ago by  Laura ESFA.
    #421855
    SFA STAFF

    Laura ESFA
    Keymaster

    Question:
    When I select ‘Download Data’ in View Education Data, the data files available are:
    Prov RO4
    In Year R12
    In Year R10
    Shouldn’t R14 be in here??
    I also have the issue of old records in there that are included in the QAR…

    Response from the ESFA

    Hi there,

    Thank you for your question.

    Prov R04 is the correct folder. It is named as such because 19/20 R04 data is used in the 18/19 QAR calculation.

    Are the old records in the data extract? The data extract contains 4 Hybrid End Years, 2016, 2017, 2018 & 2018.5. The only data that counts towards the 1819 QAR calculation are those with a Hybrid End Year of 2018.

    Thanks,

    Laura

     
    #421860

    Luke Burns
    Participant

    I am attempting to understand how the ‘Excluded’ count is calculated for Education & Training records on the QAR Summary Dashboard.

    I have looked at the provisional R04 dataset published by the ESFA, but am unable to see how the values on our dashboard were generated based on this data – it would therefore be helpful for our reconciliation to understand how the count of excluded E&T learners for 16-18 and 19+ was generated.

     
    #421862

    KLF2015
    Participant

    Is there any documentation that explains what each column actually is, and what is being counted and how? i.e. Plan_Break_Timely_Excl and what 1 means and what 0 means.

     
    #421878

    Luke Burns
    Participant

    Yes, the technical specification provides definitions for these fields, but unfortunately this hasn’t helped clear up the discrepancy in numbers.

    https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/856548/ET_QAR_technical_specification_201819_v3.pdf

     
    #421884

    lesleyvalentine
    Participant

    I am reposting this

    We have 2 students who were 15 at the start of their course – they have been included in the QAR but have not been counted as a leaver or achiever although they both got good grades.

    Lesley

     
    #421888
    SFA STAFF

    Laura ESFA
    Keymaster

    Hi Lesley,

    Sorry for the delay – we are investigating your query. As soon as I have a response I will post it on the thread.

    Best,
    Laura

     
    #421890
Viewing 25 posts - 26 through 50 (of 62 total)

You must be logged in to reply to this topic.