SIS means Service Information System. It is managed by LogicalOutcomes, a Canadian non-profit that provides evaluation and consulting services. SIS is a tool that we use for our own evaluations. It has several elements:
- A private site where you can manage your evaluation tools, send out surveys and analyze your data.
- A library of survey instruments, interviews and other data collection tools customized for your programs.
- A growing registry of evidence-based measures and indicators that you can add to your library.
- A powerful interactive data visualization tool that allows you to explore your data in depth and communicate results to your stakeholders.
Contact us to schedule a conversation. SIS is complicated and kind of hard to explain in writing.
The measures that SIS uses in its surveys and data collection tools are free or open source. You can copy and use them in your own systems as long as you comply with their licensing terms. The instruments we create ourselves can be copied and reused under Creative Commons licensing. SIS charges fees for the effort to code metadata and design visualizations in Power BI. Costs vary from about $100 upward per instrument depending on complexity.
You may not need to add instruments. We have created standard surveys and other data collection tools that can be customized for your own programs without extra cost.
The Basic and Group SIS subscriptions are designed to be used by small non-profits and people without specialized evaluation expertise. SIS subscribers just need to know about confidentiality and ethical research practices to protect the privacy of vulnerable clients. We will provide training on request, or we can handle the whole evaluation for you.
The expert subscription is for experienced evaluators or larger organizations that want more control over their evaluation tools. Experts will be able to create their own reports using Power BI and analyze raw data using their own tools (such as R or SPSS).
If you have an Expert Subscription you can add your own questions, create your own surveys and design your own data visualizations.
About SIS subscriptions
SIS has many different elements, but to use it as an evaluation platform you need an ongoing subscription. A subscription allows you to send out surveys, collect data, analyze it and store it.
Without a SIS subscription you can still browse measures and use them in your own evaluation systems (assuming you meet the licensing terms). You can also download and use KoNote. But you can’t collect and analyze survey data.
A Basic Subscription includes everything you need for a small evaluation, including up to five instruments that you can customize for individual programs and groups.
A Team Subscription offers additional users, responses and instruments. You may want to invite several staff members to interact with the data visualizations for example.
An Expert Subscription is for consultants, experienced evaluators and/or larger organizations. You get access to raw data and the ability to create your own visualizations, along with additional users, instruments and responses.
Yes. You may find that you need more users, or fewer than you expected. You can switch with 30 days’ notice as long as you are not using more resources than your subscription can handle. If you need more instruments (e.g., if you already have 5 and you want a few more on a Basic Subscription), you can just pay for the additional instruments.
About indicators and measures
SIS will draw measures and indicators from many sources, including registries like LOINC, CYFAR, PHENX, the National Quality Forum and many more. Future blog posts will describe some of our main ones. We also want to add measures that agencies use and like from their own research. If your favourite measures are supported by evidence, let’s add them to SIS.
The evaluation field is dense with jargon, and to make it worse, technical vocabulary is unstandardized. If you ask a dozen evaluators to define terms like logic model, theory of change, measures, indicators and instruments, you’ll get scores of different answers.
SIS defines them as follows:
- Instruments are data collection tools like surveys, assessment forms and interview protocols. Instruments include one or more measures (see below). They also include elements like introductions, consent forms, privacy policies and submit buttons. They may be online or paper-based, and could be applications in their own right. They could even be spreadsheets that are used to import data into SIS.
- Measures are the questions that collect the data you are looking for. For example, a demographic measure might ask, “What is your age?”. Or an outcome measure might ask many questions about your current health status. A single measure can be as short as one question or as long as hundreds of questions (I’m thinking about psychological tests like the Minnesota Multiphasic Personality Inventory (MMPI) which has 567 questions). We call it a measure because it measures something that we want to find out. In SIS, you can choose measures that can be added to instruments.
- Indicators are the summaries that help you to understand your program. For example, you might want to show your funders that your program primarily serves young women. Your indicator might be percentage of clients who identify as women (using a measure of gender) between 18 and 34 (using a measure of age). A simple way to differentiate measure from indicator is that an indicator is a calculation based on a collection of measures. In SIS, indicators are generally shown in data visualizations using Power BI.
‘Validated indicators’ mean indicators that have been shown, through research, to predict the outcomes or concepts that they are tied to. It can cost millions of dollars and many decades to validate an indicator. For example, ‘BMI’ or Body Mass Index is a health indicator that can predict many risks, including diabetes, heart conditions and mortality. It turns out that it’s not as accurate an indicator for mortality as Waist Circumference or Waist to Hip Ratio, but it’s probably easier to measure (see https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4120111/). Choosing good indicators isn’t easy — you have to find a measure that is evidence-based, appropriate for the population you are assessing, feasible to collect, and credible for the stakeholders you want to influence.
We provide a starter pack of surveys and interview protocols that focus on gathering useful information from clients and staff. We will be adding more instruments to the library as organizations request them, and they will be made available to everyone.
The measures that SIS uses in its surveys and data collection tools are free or open source. Measures themselves are always no charge, because we don’t own them. (Actually, we do develop some SIS measures, but they are freely available under a Creative Commons license.) You can copy and use SIS measures in your own systems as long as you comply with their licensing terms.
The instruments we create ourselves can be copied and reused under Creative Commons licensing. SIS charges fees for the effort to define the metadata, create the instruments, and design visualizations in Power BI. Costs vary from about $50 upward depending on the complexity of the instrument.
KoNote is an open source software program that is offered as part of SIS, but can be downloaded and used on its own. It is encrypted and stored on individual computers (Windows, Mac and Linux), not hosted on the Cloud, so it is protected from online attacks. KoNote can collect personal data and manage clinical notes for your programs.
SIS uses KoNote as a data collection tool. KoNote can export data (stripping names and personally identifiable information) into SIS to enable longitudinal analyses of program results.
KoNote was developed by a Canadian psychiatrist and software developer, Dr. David Gotlib, for use as an Electronic Health Record. Dr. Gotlib is a Board member of LogicalOutcomes and donated the software code to our organization in 2018.
We designed SIS to be multilingual (even the acronym ‘SIS’ works in French, Spanish, Portuguese, Italian, Romanian and other languages). The administrative back end (e.g., metadata definitions) and reports will be English-only until we can get funding to expand languages, but surveys, interviews and other data collection tools can be easily translated into multiple languages.
We plan to begin with French and Arabic. Many of the validated measures in SIS already have high quality translations available, and we just need funding to add the appropriate question texts into our metadata tables. Ask us for details.
About reports and data visualizations
We work with experienced data analysts and Power BI specialists to design charts for each SIS measure. Then we design reports for entire instruments. See the Gallery for examples.
If you want additional charts for a measure or instrument, we would be delighted to work with you on the design. We charge just enough to cover our costs, and then if the design would appeal to other users we add the new charts to SIS.
About KoNote client tracking
KoNote is a standalone software program that supports Windows and Mac computers. It can also be used on a mobile tablet using a browser and a virtual Windows terminal.
It does not need web hosting. In fact, it was designed to be used on a local network drive in a hospital or social service organization. It can be shared on a Dropbox or OneDrive folder, or just on a single computer.
KoNote takes only about 5 minutes to install, but much longer to configure for your program. You need to define program goals, outcome measures, progress tracking and so on before it’s ready to use.
We plan to create a couple of basic (and free) templates for quick setup. If you need help we can also provide consulting and technical support.
About working with us
SIS is designed to be used by people without specialized evaluation expertise. However, SIS administrators must have some knowledge of privacy and ethics to protect the privacy of vulnerable clients. We will provide training on request.
SIS was developed by a team led by Dr. Gillian Kerr, an organizational psychologist with three decades of experience in evaluation and policy analysis. She wanted to dramatically reduce the cost and complexity of evaluation for nonprofits while improving data quality and security. SIS was originally developed in partnership with the Ontario Council of Agencies Serving Immigrants (OCASI), funded by the Ontario Ministry of Citizenship and Immigration.
Actually, yes, sort of. SIS is run by LogicalOutcomes, a Canadian nonprofit. We are looking for partners and evaluators who are interested in using or contributing to SIS, either as a consultant or volunteer.