Level Four Advancement and Maintenance

There hasn’t been a Level Four Magic Judge since 2016, but starting soon, we will make some new ones. The old L4 was very similar to Grand Prix Head Judge, but the new L4 is not. This level is another stepping stone between what the old L2 was, and old L3. The immediate target audience for this level is certainly smaller than Judge Foundry Level Three, but we do expect quite a few quick advancements. Continued feedback through JudgeApps, Reddit, Facebook, Twitter, our new Discord server, or by old-fashioned email is still highly appreciated.

A Level Four Judge is an expert in tournament policy with experience head-judging events of 150+ players and small teams of judges. L4s are also comfortable leading a large team, like a 5-judge team at a Regional Championship. Not only are L4s comfortable with all of the tasks expected of a floor judge at a large event, they are experts with at least one team and can lead any team while training L2s and L3s on how to do those tasks. L4s are strong communicators and mentors. They train L2s to become L3s and mentor L3s to become L4s. In their local community, an L4 identifies judges who should be advancing, works one-on-one with judges, and encourages mentorship and collaboration within the entire community. An L4 should start looking towards L5 when they find themselves very comfortable in all of the L4 roles, mentoring multiple other judges towards L4, and are highly involved in the regional and national community.

Level Four looks a lot like what many judges wanted to call “Level Two Point Five” for a long time. It’s a combination of L2 Tester and the various large team lead badges along with the ability to head judge larger competitive events. Many L2s circa 2012-2019 found their way to old L3 by being a “red shirt” at a StarCityGames Open/Tour/CON event as the Sunday 5k Head Judge, or got their shot as a Super Sunday Series Head Judge at a Grand Prix. That step up is what Level Four represents: a mainstay judge at many multi-day events, respected by the community, and strong head judges for NRG Sunday 5ks, F2F Tour events, and Secret Lair Showdown events at MagicCon. 

Advancement

Candidates for Level Four first need to deepen their skills at additional multi-day events. Like other levels, there are some specific requirements, but promotion to L4 is not about checking boxes. It is a journey that will require reviews, recommendations, and a panel process. 

To be certified for L4, a candidate must complete the following items.

1. Work Events as a Level Three Judge

Like for Level Three, nothing on this list is meant to stop someone from advancing, but rather as a guide for the path between a new L3 and L4 competency. Even though this list may look shorter than the list for L3, and there isn’t a time requirement, there is an expectation of learning and growth along the way. Requirements listed aren’t there as hurdles, but to ensure that candidates experience each role than an L4 would commonly have. We recognize that organized play is always changing and requirement difficulty might ebb and flow. If one requirement ends up being onerous for a candidate, they should work with the testing manager to identify an alternative that fits the philosophy of it, and we’ll consider if this also merits a universal change.

a. Work at least six multi-day events

b. Lead a team that is assigned each of these tasks at least once

i. Pairings

ii. End of Round

iii. Deck Checks

iv. Scheduled Events

v. On-demand Events

c. Lead a team with 3 or more judges reporting to them at least 3 times

Once a judge is certified as L3, they are likely to begin leading teams more and more often. Through that process, the candidate should lead each team at least once, but more than likely, multiple times. L4s are expected to be experts in at least one team, and competent at all of the teams. There isn’t a time limit requirement either; many candidates will take their time, and their requirements will be respected no matter how long has passed.

d. Serve as head judge for a headlining event at a multi-day event

A headlining event at a multi-day event is like a Sunday 5k, afternoon Legacy 5k, or NRG’s Sunday event. The candidate isn’t expected to be the top billing head judge of the weekend, but to head-judge one of the other major competitive events. As explained in the L3 article, this event likely has a couple of teams, and 4-7 floor judges. If there is any doubt about whether an event qualifies, reach out to Judge Foundry and we will clarify.

e. Work at a competitive event with at least 20 other judges

It might be possible to head judge an event and meet the previous requirement without attending a multi-day event with a very large staff, and L4s need to have some understanding of the highest scale of events. Some regional tournament organizers may never reach this scale; it is expected that L4 candidates are working events with multiple TOs and at the largest scale.

2. Write at least three reviews in the previous year, as an L3

a. Must be from events that the candidate and the other judge both worked

b. Must include at least one review of an L4+ judge

c. Must include at least one review from a multi-day event

d. Each review must contain detailed and actionable feedback.

The review quality level bump from L3 to L4 isn’t massive, but the expectation is that the candidate is able to provide deeper feedback based on their increased experience. A panel is likely to use the candidate’s reviews to evaluate some of the qualities.

After completing at least those requirements, candidates for L4 have two routes to advancement. 

One major piece of feedback received in the early days of Judge Foundry was that the old Level 3 process was highly dependent on the candidate’s ability to get recommendations from other judges. Two options here help alleviate that possible roadblock by providing candidates a way to self-promote if that’s more appropriate for them. The candidate must receive approval to move onto the L4 panel process based on a review/recommendation. 

1. The candidate can work with an L5 mentor until that mentor is comfortable with their assessment that the candidate is ready for L4. Then the L5 writes a detailed review of the candidate, explaining how the candidate meets the requirements for L4. The review will be assessed by an approved L4 panel lead, who may advance them to the L4 panel process, or return the review to the L5 with feedback.

The mentor mode puts onus on an L5 mentor to do a lot of writing and promoting of the candidate, and may be more appropriate for candidates who struggle with self-promotion. 

2. The candidate can submit a self-review indicating their preparedness for L4 according to the following requirements. The review will be assessed by an approved L4 panel lead, who may advance them to the L4 panel process, or return the application with feedback for the candidate.

The self-review mode is entirely self-driven and may be more appropriate for judges with strong self-evaluation skills who aren’t working directly with a single mentor, or who’s geography make getting a single extensive review difficult.

L4 Panel Process

Once approved, candidates move into the L4 panel process, which is managed by the Advanced Testing Manager. The testing manager manages both L4 and L5 processes, and works out logistics for evaluation and panels at events and online.

Advanced Rules/Policy Exam

After being approved for the L4 panel process, the candidate may take the Advanced Rules/Policy exam. They must complete the exam with a score of 70% or higher before a panel can occur.

This exam extensively tests the candidates rules and policy knowledge. L4 is the level where we expect knowledge of these topics to plateau, so candidates need to pass the exam. The exam can be taken immediately, and must be taken before a panel starts. Candidates passing a panel and then failing the exam is undesirable in multiple ways, so the exam is required first. 

The passing score for the exam is 80%, but candidates with scores between 70% and 80% don’t automatically fail. Depending on the panel lead’s evaluation, a candidate that meets expectations in Tournament Policy Philosophy (TPP) with a score in the mid-70s might be evaluated as having an area for improvement in Game Knowledge. A candidate with a score of 78 who exceeds expectations in TPP is likely to be evaluated as meeting expectations in Game Knowledge.

Panel Selection

After being approved for the L4 panel process, the Advanced Testing Manager will choose an L5 approved to lead L4 panels as the panel lead. The panel lead and testing manager then choose at least one additional panelist. 

The panel lead generally can’t be the L5 that wrote the L4 assessment for the candidate. The panel lead is granted reasonable autonomy in selecting additional panelists, who may be L4 or L5, but should be approved by the testing manager.

L4 Practical Assessment

The assigned panel lead may request that the candidate complete a L4 practical assessment. 

After reviewing the candidate’s L4 assessment, and talking to panelists, the panel lead has the option to require a practical assessment. Strong candidates may bypass this stage if the panel lead knows their skills very well, and is comfortable moving to the panel without additional information from the practical. Most candidates should expect to need to complete a practical assessment.

The assessment happens at a multi-day event where an evaluator and the candidate are both on staff. The evaluator observes the candidate’s proficiencies, leadership, and success at the event and writes a review, assessing the candidate. The evaluator provides the review to the panel lead. 

The panel lead, a panelist, or another chosen evaluator does the observation. Everyone involved works with the testing manager and tournament organizers to identify an appropriate multi-day event where the candidate and an appropriate evaluator is on staff.

The review should cover a broad range of skills expected from a Level Four Judge, including, but not limited to:

1. Pre-event communication
2. Preparation for team or head judge tasks
3. Communication with other judges, leads, and head judges
4. Success at team tasks
5. Mentorship

Evaluators are strongly encouraged to include any assessment or feedback relevant to the L4 skillset. 

The review should extensively cover anything that the panel might be interested in. The evaluator isn’t expected to be the final judge on whether the candidate is prepared for L4, and a weak performance doesn’t doom a candidate’s chance at passing a panel. The evaluation is intended to be a slice of the candidate’s work, and be used to inform the panel’s assessment. 

L4 Panel

The Testing Manager, candidate, panel lead and panelists will coordinate an appropriate time and venue for the panel. In-person panels are highly recommended, but online panels may be considered in circumstances where an in-person panel would be extremely difficult to schedule. 

The testing manager is involved again to set up logistics. We don’t know what the cadence of L4 panels will be and whether online or in-person panels are more likely, time will tell as some are scheduled next year. It’s possible for a panel lead to schedule the practical and hold the panel the same weekend; if that’s the case it’s highly advised that the panel lead not be the primary evaluator for the practical, to balance both workload and different sets of eyes evaluating the candidate.

The panel is a group interview where the panel asks questions of the candidate to assess them in each quality. The questions will vary, depending on the candidate’s known strengths and weaknesses. 

Some judges may be familiar with interviews for previous badges and certs, but not familiar with the way that previous Level 3 panels were run. L4 panels are expected to be similar to old L3 panels, but take up less time. The necessary quality bar for L4 is lower than old L3, so panel leads won’t need to dig as deep on some qualities to be confident the candidate meets the requirement. It’s expected that most L4 panels take less than one hour, accepting that some where the panel lead did not require a practical might take a bit longer.

The panel lead will assess the candidate in each of the 4 quality categories, including each subcategory. 

Each category will be evaluated and assessed on this scale.

1. Exceeds Expectations
2. Meets Expectations
3. Area for Improvement
4. Deficient

Candidates who are evaluated by the panel lead during the panel to meet expectations in a majority of categories, with no deficiencies, are promoted to Level Four at the panel.

While all levels have their minimum floors for skills and qualities required, L4 (and L5) are evaluated differently. L4s are expected to meet a minimum quality requirement across each category of qualities – none must be evaluated as deficient, so far below the expectations of an L4 that alone it results in a failed panel. Additionally, candidates must also meet the general quality expectations of L4 by meeting expectations in a majority of categories. 

L4 panels are additionally regulated by the Level Four Panel and Testing Guide, maintained by existing L4s and L5s. Portions of the requirements are included here as examples. Generally, the L4 process is designed to assess whether the candidate: 

1. Is capable and comfortable head-judging a competitive event with 150+ players and small teams of judges.
2. Is capable and comfortable team-leading a team with 5 judges at a multi-day event
3. Is an expert in the logistics and philosophy of at least one team’s tasks, and capable and comfortable with each other team’s tasks
4. Is capable and comfortable mentoring judges to become L3
5. Has strong personal skills that enable head-judging, team-leading, mentorship, and participation in the community

The panel and testing guide isn’t a public document, because it will be maintained to make panels consistent across the future of Judge Foundry. The testing guide is the brainchild of every approved L4 panel lead, documenting questions and panel strategies for future panels to use. Only L4s and L5s will have access to the guide, to prepare for future panels of which they may be a part.

L4 Quality Requirements 

This is a brief description of how an L4 candidate might be evaluated on each quality. This is not exhaustive, but serves to give examples of how candidates might be evaluated and what general expectations are. This section replaces the Skills section from the other levels, as it also describes in detail what skills L4s are evaluated to have.

• Game Knowledge
  • Rules 
    • Tested by the Advanced Rules/Policy Exam
    • With the exception of sections 801-809 and 811, the entire Comprehensive Rules may be included on this exam. Candidates should have a clear understanding of the rules of the game and be able to articulate its building blocks from memory
  • Tournament Policy Application
    • Tested by the Advanced Rules/Policy Exam
    • The entire IPG, MTR, and JAR may be included on this exam. Candidates should be able to answer questions about infractions, penalties and remedies from memory, including application to situations not directly described in examples, and to select the most applicable of each for described situations
    • Knowledge of the Digital MTR or other community supplemental tournament policy will not be tested
  • Tournament Policy Philosophy
    • The candidate must show strong understanding of the underlying philosophies that inform the MTR, IPG and JAR 
    • The candidate must show some ability to articulate how these philosophies are applied, and how they result in the written policy
    • A deficient candidate applies policy or its philosophy incorrectly, and may not understand the appropriate times to deviate from policy, or why not deviating is important
• Event Skills
  • Tournament Operations Proficiency
    • The candidate shows good knowledge of each team and task utilized at large tournaments
    • The candidate shows knowledge of product distribution logistics, and limited procedures
    • A deficient candidate might not understand how one or more teams work, like forgetting that the paper team should make sure that pairings boards are in place before the tournament starts
  • Tournament Operations Philosophy
    • The candidate shows good knowledge of tournament philosophy, and the ability to balance results with the time cost involved
    • A deficient candidate cannot provide alternative solutions for problems that might occur at a tournament
  • Investigations
    • The candidate can identify instances for potential cheating, and doesn’t overlook them in judge calls
    • The candidate can explain the difference between opportunistic and premeditated cheating
    • The candidate can make a plan for talking to a player during a cheating investigation, operate that plan, and adjust based on new information
    • A deficient candidate may overlook the potential for advantage in an infraction or may make decisions based on “gut feel” rather than evidence
• Leadership Skills
  • Team and Event Coordination
    • The candidate communicates appropriately with head judges, team leads, and other judges in preparation for and at events
    • The candidate coordinates team tasks with their team and with other teams
    • A deficient candidate may isolate themselves as a team lead, have difficulty communicating with other leads, be unable to delegate tasks, or unable to teach tasks
    • A deficient candidate may also be unable to manage a team while maintaining team morale and mentorship
  • Mentorship
    • The candidate is capable of mentoring judges to achieve Level Three, including performing L3 interviews
    • The candidate understands the definitions of L1, L2, and L3, and is able to appropriately evaluate judges based on those definitions
    • A deficient candidate may be unable to identify any meaningful weaknesses in their peers, or unable to provide critical feedback
    • A deficient candidate might not understand the requirements for L3, and might not be able to appropriately fail a candidate’s L3 interview if the candidate isn’t prepared
• Personal Skills
  • Conflict Management
    • The candidate is capable of handling conflict, both involving themselves and between other judges
    • A deficient candidate may concede their position to their own detriment to avoid conflict or be unprepared to deal with a conflict between players
  • Diplomacy
    • A strong candidate is mature, trustworthy, amiable and well respected by their peers
    • A deficient candidate may have trouble working with others, or often fail to maintain decorum, diplomacy, and tact, either in person or in an online setting
    • A deficient candidate may have accumulated a trail of other judges who don’t like working with them, and is unable to resolve any of the issues
  • Self-Evaluation
    • The candidate understands their strengths and weaknesses, and works towards growing in places in which they aren’t proficient
    • A deficient candidate’s self-reflection lacks accuracy or depth, and the candidate may not put effort into actively improving where they are struggling
  • Maturity
    • The candidate is understanding of others, punctual, and understands and embodies professionalism
    • A deficient candidate may be often regarded as negative, tardy, irritating, difficult to work with, and might favor complaining about a problem over and over rather than trying to find a solution
  • Stress Management
    • The candidate is capable of dealing with stress, and understands how they operate in a stressful environment
    • A deficient candidate may fold under pressure, or actively avoid stressful situations to their own detriment
  • Teamwork
    • The candidate works well with a team, and knows their place within leadership structures as they change event-to event.
    • An excellent candidate thrives as a team member, bringing up the morale and teamwork of the entire team, while making their lead look good
    • A deficient candidate might have trouble taking directions, or trying new things that a lead asks them to do. They might also attempt to take over from an inexperienced lead instead of letting them grow

Maintenance

Level Four introduces additional highly-customizable modal options for completing maintenance. L4s are expected to be more involved in the community, and some years might spend far more time supporting projects than judging events, but are still expected to remain up-to-date on rules. 

To maintain the Level 4 certification, a judge must complete the following items each year

1. Choose one —

• Pass three out of four advanced set update quizzes throughout the year
• Pass an Advanced Rules Practice and Advanced Policy Practice test

The update quizzes for L4 are different from L2 and L3. They feature more advanced questions, and expect higher level judges to understand detailed changes to the rules, and specifics about new mechanics. Practice quizzes are designed to prepare L3s for the Advanced Rules/Policy exam, and are likely more challenging than the update quizzes.

2. Choose three —

• Lead a total of 20 judges as a Team Lead at events
• Head-judge an event with at least five judges
• Work at least six multi-day events
• Serve on one or more L4 advancement panels
• Lead a core project
• Create educational content (ex: article, conference presentation, video, etc.), subject to approval

L4s have even more flexibility with how they either work at events or contribute to the community to complete maintenance. 

It’s highly suggested to get educational content ideas pre-approved by an L5 so that lots of work doesn’t get poured into an idea that might struggle to get approval post hoc. 

3. Choose one —

• Write a self review
• Receive an in-depth review, covering multiple events, from an L5

Like L3s, L4s have the option to document their self evaluation, or work with a mentor. These self reviews aren’t expected to be as extensive as those submitted to apply for a new level. Reflecting on events worked, improvements, and goals for the next year is sufficient.

4. Maintain membership in Judge Foundry by being up-to-date on their membership dues

Benefits

Certified Level Four Judges receive the following privileges as members of Judge Foundry

1. Use of title Level Four Judge or Judge Foundry Level Four
2. Access to private Judge Foundry L4 resources, forums, and chats
3. Voting in Judge Foundry leadership elections
4. Ability to certify Level Three Judges, and serve on Level Four panels

Postscript

Just like the Level Three article, some judges are going to read this and think “I’m already there, I should be L4 right away.” That’s also the intent for L4. The process for those folks will be different from L3, as a single L5 can’t get them there on their own, because of the existence of the panel process. 

Judges who are clearly Level Four Judge Foundry Judges will have an opportunity to achieve that level in an expedited process, which will be explained in the near future. The best path forward for those judges is to get their L3 cert completed quickly. Then they should think about all of the qualities from this article, and how they would best describe themselves as already functioning well above the standard for L4. 

There are also judges who are clearly Level Three, and on their way to Level Four. It won’t hurt them to attempt the expedited process, but we believe there will be several L4 panels in the next year for judges who aren’t quite ready to be fast-tracked to L4. We’re excited to see how these new panels develop, and want to see judges move through the process so they can be promoted, and we can get feedback and refine as well.

As for current Level 3 Judges, certified under the old Judge Program or under Judge Academy: we have a plan for an expedited process to allow you to sort yourselves into L4 and L5 as well. The requirements for the new L4 are generally below the bar to be promoted to the old Level 3, so we want to make this a fairly quick and painless process for judges who are still active. We’ll be ready to talk about that process soon, after we’ve shared Level Five with you next week.