Is the HP programme well supported by the executive stakeholders, funders, and project partners?

• Is HP considered and used as the single source of truth?
• Do project partners consider HP to be an enabler of system change?
• Does the HP programme have executive champions?
• What do the stakeholders and funders perceive the value of HP to be?

• Is HP being used as a tool to support broader system change?
• Is HP an integral part of the service planning process?
• Support commitment from project partners for:
  • Successful engagement and integrations
  • Continued funding
  • Contribution to and endorsement of HP as an integral part of service planning and process
  • Feedback on and endorsement of the long-term goals, including evaluation

• Review feedback
• Continued executive and project partner support for:
  • HP programme
  • Engagement
  • HP integration with the wider health system

• Governance committee meeting minutes
• Interviews with governance committee members
• Feedback:
  • HP programme team
  • Clinical and non-clinical leaders
  • Key personnel

How does the wider health system view and use HP?

• Is HP considered and used as the single source of truth?
• Is HP accessible to, and supported and used by the wider health system?
• Is HP viewed as a tool for communication?
• Have relationships improved between primary, secondary, tertiary care?

• Identify opportunities for increased collaboration between primary, secondary, and tertiary care
• Identify any programme barriers
• Determine the usefulness of HP as a vehicle for health system communication and relationship building
• Establish the level of health system support from:
  • Key leaders
  • SMEs
  • NGOs
  • Allied health
  • Clinicians
• Identify participation in and engagement with the HP programme

• Review feedback on HP's influence on the wider health system
• Engagement:
  • SMEs with CWGs
  • SMEs with drafting process
  • Allied health

• Interviews with governance committee members, and key clinical and non-clinical leaders
• Feedback:
  • CWGs
  • SMEs/CEs
  • Allied health
  • Clinicians
  • Dot
• Surveys

Are the HP programme objectives being met?

• Track programme indicators over time:
  • Localisations
  • Reviews
  • Development/drafts
  • Partial updates

• Number of pathways or pages in development or draft, localisations, reviews (12 categories)

• Dot

How is the HP programme utilising its resourcing (time)?

• Is the programme efficient?
• Are some specialties more resource intensive than others?

• Monitor time usage 
• Identify time-consuming specialties

• Review feedback
• Time per specialty per month
• Time per task per month:
  • Development
  • Reviews
  • Localisations
  • Partial updates
  • Meetings and correspondence
  • Site administration

• Dot
• Feedback:
  • SMEs/CEs
  • CWGs
• HP programme team records

• Identify any barriers in specialty development:
  • Time taken for specialty development, localisations, and reviews
  • Page views per specialty
  • High-resource (time) low-use specialties

• Review feedback
• Time taken per specialty for development, localisation, and review
• Page views per specialty

• Dot
• Google Analytics
• Feedback:
  • SMEs/CEs
  • CWGs
  • HP programme team

Has the implementation of HP been successful for:

• the programme
• clinicians

• Are clinicians satisfied with the HP programme?
• Identify any programme barriers or constraints?
• Is the programme working as intended?

• Review feedback

• Focus groups
• Interviews
• Surveys
• Feedback:
  • Governance group
  • SMEs/CEs
  • CWGs
  • HP programme team

Are there any service redesign, clinical improvement, or localisation opportunities?

• Identify and adjust work development priorities
• Update priority plans based on user feedback 
• Identify service redesign opportunities
• Reduce unlocalised page usage

• Page views:
  • Localised
  • Unlocalised
• Search terms
• Review feedback

• Google Analytics
• Feedback:
  • CWGs
  • SMEs/CEs
  • Clinicians

• Enlist SME and primary care involvement in pathway localisation

• Estimated number of contributors by pathway specialty
• CWG feedback

• Dot
• HP programme team records
• CWG feedback

How do GPs perceive HP?

• Is the HP programme creating value for end users?
• What improvements could be made from a GP perspective?

• Establish the level of support from GP users
• Identify areas for targeted improvements

• Review feedback

• Clinician feedback:
  • Interviews
  • Focus groups
  • Surveys
• Outreach activities:
  • Education events
  • Practice visits

Is health professionals's knowledge and use of, and engagement with HP increasing?

• Is site activity increasing?
• Are your engagement activities effective?

• Increased participation in and engagement with HP
• Increased user knowledge of and confidence in appropriate care and referral services available locally

• Positive trends in number of:
  • Users
  • Sessions
  • Page views

• Google Analytics

• Improved user experience
• User feedback on HPs impact on the quality of consultations

• Focus groups
• Interviews
• Surveys

• Engagement, education, and outreach activities

• Number of communications
• Estimated reach of communications

• Google Analytics and campaign tracking
• Feedback

• Maintain continued usefulness of HP and use it as a tool for communication and relationship building

• Review feedback
• Number of CWGs

• Feedback
• CWGs
• SMEs/CEs

• Review and log all feedback

• Feedback:
  • Dot
  • Engagement activities
  • News stories
  • Events

Are the engagement activities effective?

• Is the engagement strategy working as intended?
• How has the engagement strategy achieved this?
• Which activities are most effective?

• Determine the extent and impact of your engagement, education, and outreach activities

• Number of communications and events
• Estimated reach of communications
• Review feedback

• Google Analytics and campaign tracking
• Feedback:
  • Clinicians
  • HP programme team
  • Stakeholders