Skip to main content

I help to build ReVIEW Talent Feedback System

The ReVIEW Team

Version 4.0.3 Released

2 min read

After three months of our most intensive design sprints we have released version 4.0.3. Its hard to believe after much of this work there are few features to brag about. 

Most of the changes were at the information architecture level and were designed for scalability improvements. Basically all under the hood changes.

We want you to think a peak at the back end would look something like this:

We fear, and often think we see the our under the hood improvements like this:

The answer is, of course, somewhere in the middle. But we work hard to ensure school districts, universities, and education consultants who have come to rely on ReVIEW Talent Feedback System see a close to flawless system.

To a user what is under the hood doesn't matter, what you want to see is:

A smart and intuitive system that works in partneship with your district, evaluators and teachers. 

The changes in version 4.0.3 (the .o.3 are minor bug fixes) include:

  • Changes to reports-Districts wanted more charts and graphs. We gave them to your.
  • Stability Fixes for scalability.
  • Academic Year Features-As our first year winds down we wanted to maker sure we had the architecture in place so schools can have a truly multiyear platform that meets the legal archiving features. 

This will be the last major release of this academic year. We are most excited about these recent changes but we have some awesome features coming in our roadmap including:

  • Revamping of the district and evaluator dashboards.
  • Customizing and creating events.
  • Better video based coaching integration.
  • More charts and graphs including Cohen's Kappa and Intra-class correlations for inter-rater agreement.

The ReVIEW Team

Improving Evaluation in the Elm City

2 min read

As Connecticut relaunches a teacher evaluation system ReVIEW Talent Feedback can help districts prepare for the greater emphasis on classroom observation.

We serve dozens of districts across the nation.

In fact New Haven Public Schools now uses our  Video Based Calibrations to help ensure teachers in the district can trust their observations. Earlier this year ReVIEW Talent Feedback System signed on with New Haven Public Schools to complete over 240 Video Based Calibrations between January and June of 2017.

Video Based Calibrations, our most poular service, provide a pre and post assessment to districts looking to improve evaluator capacity. Adminsitrators watch a video from our professional development library and then score it against a district framework. New Haven will then get a report on their inter rater agreement to check for validity and reliability of their system.

But we do not stop there. Simply measuring administrators is not enough. Our unique approach to video based calibration actually coach administrators on how to observe and provide feedback.

We work with ReVIEW’s team of coaches to provide New Haven evaluators and new administrators with comprehensive narrative reports using the six standards of the ReVision Learning Supervisory Continuum to analyze feedback provided to teachers. This data will support the development of an on-going professional learning plan driven to ensure quality observation and feedback practice forevery teacher in New Haven Public Schools.

Patrick Flynn, co-founder of ReVIEW Talent Feedback System stated, "New Haven was an early adopter of newer evaluation practice and have recognized the need for continuous learning for all its evaluators. Their decision is commendable. ReVIEW provides provide evaluators at all levels of implementation with impactful feedback to support a deepening of their learning."

Greg McVerry notes that "We have made user center design the heart of ReVIEW Talent Feedback System. The coaches and evaluators of New Haven Public Schools will get the same high quality narrative feedback we expect them to deliver to teachers."

To learn more about our video based calibration for yoru district email review@reviewtalentfeedback.com

 

The ReVIEW Team

Announcing ReVIEW 3.0 with Video Based Calibrations and Group Management

4 min read

Trying to focus teacher evaluation on capacity building and not compliance is our mission at ReVIEW Talent Feedback System. We made some recent overhauls of our information architecture and released some really cool new features on 1/9/2017. This update required a ton of work and we are very proud of the newly released platform.

In a little under a year we have become one of Connecticut’s fastest growing and most successful educational technology companies.

Our growth has more to do with our mission of increasing instructional capacity and engineering better teachers than it does with our app but the services we provide to school districts would not be possible without a tool to enhance teacher evaluations.

Video Based Calibration

Our biggest enhancement is opening up our Video Based Calibration models to school districts. Districts now have the option of purchasing ReVIEW Talent Feedback System as a comprehensive tool or they can choose our wildly popular video based calibration models.

Video Library

Evaluators in a district receive access to a video from our library. These videos have been normed against numerous frameworks used in Connecticut. For districts out of state we can norm the videos against your rubric or provide you with inter-rater agreement among your evaluators. We then provide a crosswalk of common evidence.

Each video also has an available transcript and an evidence guide of effective practices.

Intuitive Interface

Evaluators then get to use our fully responsive and intuitive interface to score the classroom video. They can score at the Domain, Indicator, and Attribute level. The framework you choose is displayed using collapsible menus which makes referencing the rubric a breeze.

Feedback that Matters

Your evaluators then get real feedback about their evaluation. Most calibration activities offered by other firms simply give evaluators a score agreement report. Using our new Video Based Calibration models evaluators receive feedback from a coach who scores an evaluator's report against the RVL Supervisory Continuum.

This allows districts to track the growth in evaluator capacity over time.

Districts receive a wealth of information about performance in video based calibration. You can download all responses for future trainings and track the growth of evaluators.

We also offer a wealth of services such as one on one video based coaching and  the development of online classes within the ReVIEW platform to help evaluators grow.

Group Management

One of our greatest UI challenges has been assigning teachers to events and forms. Every district has a unique teacher evaluation plans. Some districts require different forms based on experience and they all require different frequencies of observation.

Getting this doable was easy. Making the assigning of frequency pleasant and pretty was not. We have tried numerous iterations but are excited to announce the availability of group management.

Every user in a district is automatically added to a default group. This means that for every form you have selected everyone will be assigned the default frequency.

Districts can then create new groups such as first-year teacher, tenured teacher, and non-tenured teacher.

Districts can then revise the frequency of events and assign forms on a group level.

Then when new users are added they can be assigned a group. Current users can also be edited to switch their groups as they grow with a district.

The new group management tool is great. While we designed the product to make on boarding easier our existing customers are finding interesting ways to apply the tool. Some have created study groups of 3-4 administrators and have them complete the same observations. Others districts have used the group management feature as well as our custom form builder to create online professional development for teachers within review.

Engineering Better Teachers

The features in our latest release will help any district grow the instructional capacity of teachers. ReVIEW Talent Feedback System is in the business of engineering better teachers. We look forward to working with you.

The ReVIEW Team

ReVIEW Talent Feedback System Version 2.5 .1 Has Launched

6 min read

We are very excited to announce the release of Version 2.5.1, our 27th iteration, of ReVIEW Talent Feedback System. Our team, in conjunction with a growing customer base of over a 1,000 users, have worked hard to towards a release that meets the customer needs. 


flickr photo shared by MyFWCmedia under a Creative Commons ( BY-ND ) license

 

Our latest release has features that make us proud. We have no code name, no metaphor for the release. All we bring to the table is the best product possible.

Version 2.5.1 Release Notes

Version 2.5.1 includes a few minor updates to improve upon 2.5. We approach our development with such agility we did not have time to publish our latest release notes before the next iteration went live.

Electronic Signature

Many districts wanted an electronic signature with a date and time stamp on submitted forms. Rather than make this a field that could be added to forms we now require an electronic signature when ever a form is completed by a teacher, evaluator, or facilitator.

Navigation Side Bar

A minor fix. We, like many others, use a bootstrap theme for ReVIEW Talent Feedback System. This includes a collapsible navigation panel on the left. In the past it defaulted to being open rather than collapsed. Users requested that the system remember their last action. The platform now does.

Export forms in PDF for an Entire School

School districts use ReVIEW Talent Feedback System for so much more than compliance with regulatory filings. Yet we also empower districts to meet the burdens of paperwork schools face. Districts can now get a collection of every submitted form downloaded in PDF form.

So many customers love this new feature. Not only can they send the requisite reports to district or state leaders but they can use in house documents to train teachers and evaluators.

Version 2.5 Release Notes

Our biggest iteration in many months contain improvements that make our customers proud to use the ReVIEW Talent Feedback System Platform.

Dashboard Improvements

You never know what users want until you have many in the system. Then the changes needed seem so obvious. The effort we put into improving the user dashboard was on of these “oh duh” iterations. We have now organized our dashboards by the events a teacher selected.

We also added more metadata to each “card” in an evaluators dashboard. You can now see the evaluators name, date and the teacher observed.

Event and Form Frequency

One of our greatest challenges has been predicting the variability in the evaluation plans of local school districts. At ReVIEW Talent Feedback System we believe in local control and district flexibility. So we wanted our platform to mold to any school’s needs.

One area of great variation has been in the frequency of observations and forms teachers must complete. Originally we built the system so the frequency of events could be assigned per teacher. All the forms associated with that event would then be assigned.

We quickly began working with districts that assign different forms based on the where a teacher fell in their evaluation plan.You can now assign frequencies globally and then edit forms on a per teacher basis.

Another issue we did not predict, and it took some careful work with our database validations, was setting a frequency to zero. Our original default frequency was one. We never assumed a teacher would go a year without at least one formal and informal evaluation. We were wrong. Many districts do not require proficient and skilled teachers to undertake a formal observation each year.

Setting the frequency of events and forms to zero was tricky for us. Originally when an event was created a database field was created, and nature abhors vacuums. Data wanted to be in that spot. We carefully rewrote our validations and focused on the UI. We are happy to report you can now set the frequency of events and forms to zero.

Exporting Data to Spreadsheets

As researchers and professional development experts we maybe nerding out the most about this feature. Districts can now export the results of any form into a spreadsheet. A row is created for each user and a column for every field.

School districts have noted the importance of this feature. They can now quickly cretae reports to meet the regulatory burdens in evaluation systems. More importantly schools can use the data to understand where they need to grow as a system. ReVIEW Talent Feedback System is a platform for continuous growth.

Greater Metadata for Statewide Clients

States now use ReVIEW Talent Feedback System as a method to ensure evaluator capacity of their new administrators. They have purchased licenses for districts across wide geographic regions. New principals then use our innovative Video Based Calibration model to improve their instructional capacity.

States wanted to compare results based on district and city. We now require this data when creating new schools in a district.

However, our commitment to protecting local data remains strong. Only clients who purchase multi-district licenses can ever access reports comparing district performances on our Video Based Calibration models.

Improved Facilitator Reports

While we provide the most innovative and rewarding approach to video based calibration we focus more on local control and local capacity. We want districts to basically make our trainings irrelevant by building up their own local capacity.

Our Video Based Calibration models have significantly improved evalutor capacity. Yet we want districts to provide the high quality feedback we give new administrators. District staff can now be assigned as facilitators and use the ReVision Supervisory Continuum to provide feedback on teacher observation reports.

To this end we have reworked our facilitator reports. They now allow district leaders to compare performance of all evaluators in a school regardless of their assigned facilitator.

Moving Forward

We have already started our next iteration. Districts want to move teachers between schools. Currently this requires deleting and recreating the account. In our next release schools can move teachers and administrators any time. This is actually a tricky issue to solve. What happens to the teachers data? Which school does it belong to? What about their evaluators data? Or the facilitator who oversees the evaluator?

We have it all figured out and have worked through many user stories to try and explore all the possibilities. Look forward to news about the upcoming release. If you want a cool code name — -well then keep waiting.

Until then remember focusing on instructional capacity is the best way to improve student acheivement.

 

 

The ReVIEW Team

Giving Back-ReVIEW Talent Feedback System Now Free for Researchers

6 min read

We built ReVIEW Talent Feedback System because we believe the  best way to improve student achievement is to focus on increasing instructional capacity.

Teachers do not grow through a collection of misguided mandates on teacher evaluation but through a culture of reflective practitioners. We built ReVIEW with this principle in mind.

Therefore we wanted to share an important development. We will now provide university and non-profit researchers with a license to use our platform free of charge.

What is ReVIEW Talent Feedback System?

ReVIEW Talent Feedback System is an evaluation platform with a focus on providing high quality feedback. We know students thrive in environments that allow for growth through critical conversations. Future educators and teachers are no different.


" href="https://flickr.com/photos/mrsdkrebs/7876968098">flickr photo shared by mrsdkrebs under a Creative Commons ( BY ) license

ReVIEW Talent Feedback System is built for formal and informal observations. Observers can watch and score learning events against a rubric of your design. Participants can submit reflections. They can also upload artifacts such as documents or images (video uploads not included in the free offer...the bandwidth and storage get expensive).

What Can I Do With ReVIEW Talent Feedback System?

Adding Rubrics

You can add a rubric of effective teaching (or writing, or tutoring, or kayaking...really). ReVIEW Talent Feedback System currently allows you to add any multi-dimensional rubric with multiple domains and indicators.

Right now all single domain or holistic rubrics are added on the back-end. Our next major update will allow for the customizing of rubrics on the front end. If you need a single item rubric immediatley uploaded just let us know and we can work with you to add it to the system (small fee may apply).

Finally make sure you are allowed to use the rubric. You can not add any rubric that is the protected proprietary data of a third party without providing us written consent.

Making Forms

The next step is to create a series of forms for participants, observers, or learners to fill out. These forms can then be scored against those rubrics. The interface of ReVIEW Talent Feedback System is built for educator preparation and teacher evaluation but the forms and events can be customized in anyway.

We offer the following forms pre-populated.

You can add unlimited forms to any event and select the frequency that participants, observers, evlautors or researchers must complete the form.

We offer the following types of form components:

  • text (single lines)
  • textarea (paragraph)
  • checkbox
  • dropdown
  • scale
  • goal-setting (set goals and choose action steps)
  • rubric (choose any of the four domains to add to a form)
  • calendar
  • overall scale score 
  • label (add almost any html tags as a read-only element)
  • domain score 
  • file upload (video not included)
  • table (participants enter data in table form)

All of the form results are then downloadable in a spreadsheet form for your future analysis. Each individual response is downloadable in PDF format.

What Are Some Ways Researchers Could Use ReVIEW Talent Feedback System?


flickr photo shared by nathanmac87 under a Creative Commons ( BY ) license

Have conversations with participants about their teaching and provide high quality narrative feedback to help teachers grow.

If you research teacher quality then ReVIEW Talent Feedback System may have features that will increase the efficacy of your data collections and interventions. 

Test to see if training student teacher supervisors in the writing of high quality feedback improves the student teaching experience.

ReVIEW Talent Feedback System also comes pre-pouplated with the ReVision Supervisory Continuum. This rubric is used to evaluate the evalutors. You can either use the rubric or remix it to design your own rubric to provide feedback to student teacher supervisors.

Track agreement among observers.

ReVIEW Talent Feedback System is already used to track the agreement and capacity of evaluators across the northeast. Researchers can create forms to have data collection centralized among teams across diverse regions. Furthermore our reporting systems will allow you to compare agreement of your observers and even conduct inter-rater trainings. Tables with percentage of absolute agreement, percentage of adjacent agreement, Cohen's Kapaa, and intra-class correlations are coming soon.

Provide video based feedback and coaching.

While video uploads are not included for free you can embed media from across the web into our forms. This allows you to provide video based observation and coaching as long as you host the data elsewhere. We also have a library of professional development videos that can be licensed for a nominal fee.

We can even run inter-rater agreement programs for your participants or researchers but this would also be a fee based service.

Video is a powerful feedback tool. We can help you think about your research design and how ReVIEW Talent Feedback System can help.

Create professional development modules.

ReVIEW Talent Feedback System can also be used as a learning platform by researchers. You could begin with backwards design and develop a rubric that would best capture the scales of quality in particpant learning. Then using the form builder create a series of tasks that would take the learners towards those outcomes.

If your professional development study does not involve final products you can always just create each learning event as a form and then download all the results. 

Do something we haven't thought of yet.

Have an idea that we haven't listed. Great. Reach out to us and let us know. We can give you access to a demo account and see if whatever you are thinking for your research study is possible.

What About Privacy?

At ReVIEW Talent Feedback System we take privacy seriously. Our database is encrypted and the only information we have access to are user logs and your forms. This is for customer support. We can help you customize the forms and track user erros.

If you want to increase your level of privacy you could use psuedoymns and dummy emails for participants.

You can even host your data on your local servers. We have built an API based app that you can hook into while keeping all of your own data. We would have to charge for a bit of custom development to get our servers to talk.

What Can I Not Do?

Use our tool with participants without first providing a copy of an approved application for human research from an institutional review board or a similar organization. You can test drive the system but not sign anyone up before providing documentation that your study was approved.

Use it for the evaluation of colleagues, student teachers, or educators outside of a research study. We charge for these services and the revenues help us ensure we can keeep ReVIEW Talent Feedback System free to researchers.

Use the data to make a determination in any hiring or grading situation. It's for research purposes only.

How do I learn more? 

Email us at help@reviewtalentfeedback.com and we can talk through a potential partnership. Our goal (as long as we can keep the lights on) is to keep ReVIEW Talent Feedback System free to researchers forever.

The ReVIEW Team

Getting Words Right

3 min read


flickr photo shared by zaniele under a Creative Commons ( BY ) license 

Confirmation emails have emerged as our first hiccup in customer sign-up. Fitting the right message into the fewest words possible is hard work.

For us at ReVIEW Talent Feedback System the issue gets compounded because our users do not initiate the sign up process. A district or building administrator uses our intuitive interface to quickly add and modify users. Once a user gets created they receive a confirmation email with a temporary password.

Many teachers and evaluators missed the email. So I spent today working on some edits.

Dear _______________________

Welcome to ReVIEW Talent Feedback System. We support teacher growth through high quality observation and feedback.

Your Email id is ___________________________.

To confirm your account please use the following link:Confirm my account

Your temporary password:________________. You will be prompted to change the password once you log on to the site.

To log on to the ReVIEWclick here

Thank you,

The ReVIEW Team

I saw a few issues:

  • The order of prompts was just off. By providing the link to confrim my account before the temporary password we were giving users the chance to navigate way from the email before reading key information.
  • The period after the temporary password. Never end a sentence in a password followed by a password. Many users included the password when they cut and paste
  • Do not assume users will cut and paste. During recetn role out I saw many teachwers writing the password down.
  • There was no context from the user who is receiving this email without prompting.

 

So I spent the morning thinking about edits and made the following revisions

Dear ,

Welcome to ReVIEW Talent Feedback System. Your school administator or supervisor has recently requested you join our platform.

We support teacher growth through high quality observation and feedback.

Your Email id is

Your temporary password:

Copy this passowrd. You will be prompted to change the password once you log on to the site.

To confirm your account please use the following link:

If you have already confirmed your email account log on to ReVIEW by clicking here

Thank you,

The ReVIEW Team

As you can see I switched the order of prompts and added an explicit prompt to copy this password. We need to think more deeply about the number of users (often greater than 20%) who hand wrote and then typed in the password. Maybe use a word generator instead of a random string.

This is not the strongest security approach but the passwords are temporary and there is no data behind the user. We will have to think about this some more.

We also wrote template emails for our various productd and services. Administrators can now send these out before user enrollment.

The ReVIEW Team

Capacity Not Compliance

3 min read

 

 

As Patrick Flynn of ReVision Learning and I traveled the country we kept running into a major roadblock. We set out to help districts improve their teacher observation plans.

Educators knew teachers thrive when they receive high quality feedback. Yet when ever we would encourage districts to base their talent management on growth and capacity we would hear, “Our tool doesn’t do that.”

Instead of actionable feedback teachers would receive a “report” of ratings with fragmented sentences of tagged evidence.

More troubling when we asked how evaluators were trained we got nothing back but blank stares.

Even with the focus on teacher evaluation in the last decade our systems have devolved into compliance not capacity.

So we built a better way forward.

ReVIEW Talent Feedback System

 

Over the past year Patrick and I have taken all we have learned about teacher evaluation and created a system that rethinks our efforts toward instructional capacity.

We need to recognize teachers not just as an instructors but also a learners, and like our students they will grow best through high quality feedback.

Growing Talent

Our feedback system, unlike other tools, focuses on talent growth rather than an inspection model.

You can not use ReVIEW to tag evidence. Instead our User Interface requires teachers to reflect on their teaching and evaluators to concentrate on practice.

Focus on Feedback

ReVIEW Talent Feedback System recognizes decades of research that feedback is central to learning. Building instructional capacity is no different.

Using our Claim, Connect, Action system teachers receive feedback that makes claims based upon the district’s chosen framework, connects these claims to evidence from observed classroom practice, and then provides actionable feedback to help teachers grow.

Building District Capacity

ReVIEW Talent Feedback System is hyper focused at the local level. We want to improve your district rather than simply meet required mandates.

To this end we have made evaluator capacity a central differentiator of our tool. We built the ReVision Learning Supervisory Continuum into ReVIEW. This research based rubric helps to ensure evaluators have the skills to actually observe teachers.

Collaborate

ReVIEW Talent Feedback System builds bridges between teachers and evaluators. We coach for growth, not check for compliance.

We spent the last decade trying to reform schools only at the policy level. This has lead to many broken practices. We see teachers as partners.

Customize Locally

ReVIEW Talent Feedback System empowers states and districts by creating a fully customizable and responsive interface.

We have pre-loaded ReVIEW Talent Feedback System with forms designed to focus on improving practice. Yet we heard time and again that current tools in the market place are too rigid or too hard to change. Our interface allows districts to customize their frameworks and forms.

High Quality Observations

When you work with ReVIEW coaches and facilitators your district evaluators receive the tools they need to improve schools through faculty observation.

Our reporting features help guide states and districts in making local decisions around professional development.