Barton Peveril Sixth Form College's Journey with Artificial Intelligence

Background and Initial Exploration:

BPSFC recognised the potential of AI to enhance efficiency, automate routine tasks, and solve complex
problems early. The college's Senior Leadership Team (SLT) met in January 2024 to discuss the
implications of AI across the college, including business operations, cybersecurity, teaching and learning.
An area of particular interest was its application to tasks that were repetitive, high error, those that often had
declining rates of productivity. BPSFC also felt that addressing these areas would enhance their ability to
staff their support services; roles that were vulnerable to retention issues as a consequence of the private
sector offering higher rates of pay for roles at the lower end of their pay scales. The college decided to first
explore AI solutions for business functions, the rationale behind this decision was that tasks across the
business services teams were those repetitive tasks (which often had decreasing productivity rates with
increasing error rates) and as such presented the “quicker wins”. The goals included reducing staff
workload, improving stakeholder experience, and ensuring value for money.

BPSFC began to explore the market for external consultants to support them in the delivery of the project
and achieving their aims. They engaged with approx 13 specialist companies to conduct "deep dives" into
their business functions. Before finally settling on two external companies, both were appointed to explore
AI-based solutions. The first, focused on creating bespoke "AI agents" by adapting existing AI tools.The
second, examined "Google-based" solutions, including moving systems to Google Cloud and piloting
Google Gemini across the entire college. Both companies carried out independent deep dives of the
business services. This case study primarily focuses on the first deep dive and the subsequent creation of
bespoke agents. The "deep dive" revealed 107 functions where AI could make a strong impact, these 107
functions could be delivered through the creation of 26 bespoke agents.

Development of AI Agents:

Through its partnership with the external consultant, BPSFC began developing a range of AI agents,
including:

• Barton AI: A bespoke front end to a LLM built within the college's Google tenancy, which would
  allow the college to implement its own safeguards and prompts to prevent misuse and bias. This
  also eliminates the need for costly monthly licenses whilst ensuring GDPR compliance. For
  example, staff could put class sets of data into the LLM for analysis whilst ensuring that the sensitive
  data remained safe and that it wasn’t being used to train any external models.
• Barton Buddy: A digital assistant for students, providing information and guidance from a range of
  sources. These included Google Classroom, Sites, Docs, slides, Gmail, Workspace and more. As
  well as API to the management information system (MIS) and the local bus company’s live bus data.
  This agent performs as a one stop shop for students, offering 24/7 guidance on college life. Which
  includes low level health and wellbeing support, signposting students to immediate internal and
  external support.
• Peveril Assistant: A digital assistant for staff, just like Barton Buddy for students. Providing
  information and guidance from a range of sources. These included policies, procedures, the
  colleges infrastructure and more. It also included the API access to the management information
  system (MIS).

BPSFC has developed 19 agents, in total, to improve efficiency, reduce human error, and address repetitive
tasks. By May 2024, 8 AI agents were ready for demonstration to staff. All agents built for BPSFC are fully
tested in a “sandbox” site (a testing environment that is isolated from a live system. It allows users/
developers to experiment, test new features, or practice without affecting the actual operational
environment) by staff and students selected to be part of the “red team”. The aim of red teaming agents is
to find flaws and vulnerabilities in an AI system. This is done in a controlled environment (the sandbox site)
in collaboration with the AI's developers.
Challenges and Risks:

Like all who embrace this new technology BPSFC faced several challenges and risks in its AI
implementation, which the college recognised and addressed proactively:

• Ethical Considerations:
  ○ Bias and Fairness: The college acknowledged the risk of AI systems perpetuating biases if
  trained on data reflecting social inequalities. This concern was addressed by building all
  agents within its own Google tenancy so that it could implement its own constraints and
  prompts to prevent misuse and bias. One such control implemented was the “100%
  accuracy” prompt built into the front, meaning that if the agent was certain of the output
  being correct, then it does not offer a response. Instead it replies that it does not have the
  accuracy to help with that question.
• Digital Equity: The college noted that not all students had equal access to AI tools, potentially
  widening the digital divide. This concern was addressed by developing both Barton Buddy and
  Barton AI as resources to be made available to all students free of charge. As both agents run on
  API access to a LLM the necessity for individual licences is removed, replaced instead by much
  smaller “token” costs, charged for what has been used.
• Consent: Most of the API access to LLMs require parental consent for users under the age of 18.
  This presents a challenge in terms of addressing parental concerns around the use of AI and then
  recording consent in a way that grants access to the relevant agents.
• AI Literacy: Many students associate all AI applications as a LLM. It was only through launching
  “Barton Buddy” to students that it became clear that many young people do not understand what
  generative AI is. Most users thought that Barton Buddy was itself a LLM, with access to the internet
  (despite it being made clear that it couldn't do that). Applications like “Snapchat AI” and more
  recently “Meta AI”, both of which are LLMs, helped compound the confusion.
• Transparency and Accountability: The college recognised the potential for a lack of transparency
  due to "black box" AI algorithms and the importance of establishing clear lines of responsibility
  (black box AI algorithms are AI systems where the internal workings are hidden or not easily
  understood, even by the developers who created them. They make decisions using complex
  processes that are difficult to interpret or explain.) Through the development of their own agents and
  the implementation of additional safeguards and controls, BPSFC was able to mitigate this risk. In
  addition, unlike commercial LLM and tools, all of BPSFC tools record the input (question asked) and
  more importantly, the output (answer given) - providing clear transparency.
• Data Privacy and Security:
  ○ The college noted that AI systems require access to personal data and the need to ensure
  the security of that data. BPSFC ensured that AI tools used with college data adhered to
  GDPR standards, including conducting DPIAs where appropriate and training staff on data
  safety.
  ○ Barton AI was designed not to produce images or videos to further restrict potential misuse.
• Pedagogical Concerns:
  ○ The college acknowledged the possibility of over-reliance on AI, which could reduce critical
  thinking, problem-solving skills, and teacher autonomy. The college put in place guidance
  and training for staff and students on the use of AI and encouraged the use of AI as a
  starting point, and not a finished product.
  ○ BPSFC understood that AI could impact the student-teacher relationship by reducing human
  interaction and affecting social and emotional development. The college emphasised that AI
  should not replace human guidance and interaction, but enhance it through “freeing up”
  more time for staff to offer that human support.
• Practical Challenges:
  ○ The college was aware of the costs associated with implementing AI solutions, including
  licensing, training, and infrastructure.
  ○ The college also recognised that some areas, like timetabling, were more complex to
  automate and required bespoke AI solutions.
  Opportunities and Benefits:
• Despite the challenges, BPSFC also recognised several opportunities and benefits of using AI in education:
• Enhanced Efficiency and Reduced Workload: AI agents like the GCSE Results Agent and
  Certificate AI have significantly reduced the time spent on manual tasks. For example, the GCSE
  Results Agent saved 93 labour hours during enrolment, and Certificate AI saved 300 labour hours
  annually.
• Improved Student Experience: AI tools like Barton Buddy provide students with easy access to
  information, including timetables, attendance, and grades. The college has also used AI tools to
  provide personalised learning experiences, tailored explanations, feedback, and resources, helping
  students to grasp complex concepts and improve their understanding.
• Enhanced Teaching Practice: Staff have access to tools to enhance teaching and learning
  strategies. AI has also supported the development of revision skills and retrieval practice. The
  college is using AI to create resources, support literacy improvement, offer feedback, and support
  knowledge checking and retrieval practice.
• Digital Literacy: Students are being educated on what generative AI is, how to use and evaluate AI
  technologies and understand their ethical implications and limitations.
• Future Readiness: Students are developing digital literacy and skills essential for the future
  workplace, where AI will play an increasingly important role. BPSFC has become the first UK
  ‘Gemini AI Academy,’ which provides staff with regular access to training and all staff have a
  premium Google Gemini account.
• Accessibility and Inclusion: By providing a LLM, the college aims to ensure that all students can
  benefit from these technologies and reduce the digital divide. In addition, the agents are developed
  to be multilingual, with a voice to text function and both light/dark mode to support the accessibility
  of the tools to a wide audience.

Implementation and Next Steps:

BPSFC implemented a phased approach to AI integration. Initial efforts focused on business functions,
before addressing teaching and learning solutions. The college made Barton AI available to staff, and has
begun to pilot access for students. The college also established an AI working group. The college
recognised the importance of red-teaming AI agents and began to trial Barton AI with first year EPQ
students and the students from selected tutor groups.
BPSFC is currently developing an “AI Essentials Scheme of Work” to inform students of what generative AI
is, the ethical considerations, its history and evolution. The college has held “AI inset days” in June and
November, providing training and development to staff on how to use generative AI both in and out of the
classroom. In addition there have been regular optional training sessions for staff to attend as part of their
continuous professional development, exploring the use of tools like Gemini, Gems and Notebook LM -
these sessions have regularly been oversubscribed.

Conclusion:

BPSFC journey with AI provides a valuable case study for other educational institutions exploring AI
integration. The college's approach has been proactive and considered, balancing the opportunities and
benefits with ethical and practical challenges. Key insights include the importance of:

• Ethical considerations: Addressing issues of bias, privacy, and accountability.
• Data security: Implementing robust measures to protect sensitive student information.
• Understanding AI: Ensuring stakeholders clearly understand the purpose and application of AI in
  each use of the technology, addressing any concerns.
• Consent: Obtaining parental consent for student use of AI requires planning, clear communication
  and a robust mechanism for recording who has given consent and subsequently enabling access to
  those students. This can be a slow process.
• Pedagogical approach: Integrating AI in a way that supports and enhances, rather than replaces,
  teaching and learning.
• Practical implementation: Phased rollout with continuous evaluation and refinement of solutions.
• Staff and student training: Ensuring everyone has the knowledge and skills to use these tools
  effectively.

By embracing a responsible and iterative approach to AI, Barton Peveril is positioning itself to provide a
more personalised, efficient, and future-ready educational experience for its students, and is contributing to
the understanding of how AI may be integrated into the educational landscape.