Program Evaluation

© 2017-2021 Walden University, LLC 1

Program Evaluation Program Transcript

[MUSIC PLAYING]

RICHARD BALKIN: As a practitioner, or even as a researcher, you're going to be doing program evaluation, because essentially, what you need to be able to do, is to be able to identify the fact that what you're doing, whatever services that you're providing, are actually effective.

In the mental health world, you want to know that your clients are getting better with the programs and services that you're using. In terms of school counseling, you want to know that your practice, your job responsibilities, add to accountability and success of the students you work with.

So, in this respect, program evaluation ties hand-in-hand to evidence-based practice. What are the models that you utilize that are known to work? Or can you show that the models that you are using actually do work?

The program I work with is the Antonio Garcia Center. It's a program that services court- referred youth. We provide family counseling services, family intervention. We also provide counseling services for the youth.

And, as you can imagine, with a variety of problems that core referred youth have– such as gang involvement, substance abuse, difficulty with school, truancy, for example, maybe family dysfunction has led to shoplifting or other antisocial behaviors– that with the variety of programs we want to make sure that the youth that we serve actually do get better. And how do we evaluate that?

Well, let's take you through a program evaluation model. This model is known as the Accountability Bridge. And in this model, we'll take you through the specific steps that move into how we know whether or not the services that we're delivering are actually effective, and maybe even contribute to evidence-based practice.

What we have with the Accountability Bridge Model is a cyclical model. You can start in multiple places on the model, but for the purposes of this exercise, we'll start with program planning. And program planning, we're simply wanting to think about what are the services that we're going to be providing for the youth in order to address a particular problem?

For example, maybe we have a high degree of family conflict with the kids that we serve. And so we're going to offer programs related to conflict resolution. Maybe we notice that a lot of our kids have problems with controlling their anger, so perhaps we have some type of program that addresses anger management.

Program Evaluation

© 2017-2021 Walden University, LLC 2

And so what we do is we plan for the specific area that we wish to address. What are the psychoeducational components that go into teaching anger management? What are the psychoeducational components that go into teaching about mediating family conflict, or even resolving it?

So, for a program planning perspective, we think about both the psychoeducational components and the process components that we want to include in the program.

The next step is program implementation. You need to consider the fact that, OK, we have these youth who were referred by the court for services. Now, how are we going to do this program?

When are we going to offer it? For example, is this a weekend program, or does this happen on weeknights? How will kids get transported to the program? How will they be able to come? How long will the program last? How many weeks will we provide these services?

Who is going to provide the services, and what training do they have to provide the services? These are the aspects of program implementation.

And then, what exactly are we doing? As I mentioned before, if we're doing a psycheducational component, actually providing the necessary materials for the psychoeducational component. Or, having qualified practitioners to process with families and youth about the problems that they may be experiencing in their home or with their families.

As we are providing these programs, we want to monitor exactly what's occurring. Do we see kids engaging, and do we see families engaging? Are we noticing any changes?

Does the information that we provide seem particularly useful, and if so, how are we assessing that? Maybe we are monitoring the program by providing some formative evaluations throughout the process, evaluations that happen across the process. Maybe they're questionnaires, maybe they're surveys, maybe they're instruments.

However, when compared to kids who also had a history of offenses but did not receive services at the Garcia Center, we notice those kids, over half of them had a history of reoffending. So, we were able to say to our stakeholders, look, our program is effective. Kids who come through our services are more likely not to reoffend. That's an important summit of evaluation.

Naturally, what we want to do now is we want to be able to bridge what occurs in our program with the context, with who's going to be utilizing the services, and

Program Evaluation

© 2017-2021 Walden University, LLC 3

how is this information beneficial. So, as part of any scholarly practice, it's not sufficient just to have these findings. We also have to get the word out.

And in our particular case, with the Garcia center, it was good news. Our program seems to be working, and we want to let our stakeholders know, hey, our program is working. So we arrange meetings with the juvenile probation offices and perhaps with the judges who are invested in using our program for court- referred youth. And we let them know that we seem to be making an impact, that kids who get referred to our program, by and large, the majority of them will not reoffend.

So, once our stakeholders are aware of the good work that we're doing, we also want to get feedback from them. Perhaps they have other problems they want us to address, or perhaps they want to make sure that we're able to continue the services that we're offering.

So, it's important for us to be able to either verbally or even in writing, or perhaps through survey, be able to find out what they feel about our program and the services that we've provided. Naturally, with the judges that we work with, they were pretty happy that they were not seeing the same kids over and over again, that the kids seem to be genuinely doing better.

What we have found working with the juvenile courts is that there's no single problem to resolve, but more many multifaceted problems to address. Perhaps we need to address graffiti, or perhaps we need to address gang involvement, or perhaps we need to address anti-social behavior, or school truancy.

So, the judges give us a lot of feedback in terms of what problems they see in the process. At this point, they begin to say, well, what services can you provide? Here's what I think we're needing. In particular, one of the judge said, we need something for gang involvement. We're having more and more youth involved with gangs. And the response from the Garcia Center is, well, let's figure out what we can do.

So, with the feedback that we get from the judges, it ties into, what are the needs the community? So, we begin to both strategize and identify what needs the community has.

With respect to gang involvement, we can see whether or not there was a rise in anti-social activity or violent crimes as it relates to gangs. We may need to address issues related to substance abuse, or we may need to address issues of weapons and gun control. There's a variety of needs that can be addressed with gang involvement. So, one question is, how are the youth getting a hold of firearms, weapons, that bring about more violent crime?

Program Evaluation

© 2017-2021 Walden University, LLC 4

So, we begin to assess the needs of the community, find out how we can address some of these problems, and hopefully figure out how we'll be able to get youth who would be appropriate for these services to participate. As you can imagine, kids who have highly anti-social behaviors may be difficult in terms of getting them into a counseling group or session. So, just seeing not only the plan, the needs the community and the needs the youth, so that they have some buy- in into the services that we provide is essential.

For us at the Garcia Center, we had to be able to identify what was realistic. One of the things that we did is we looked at model programs to identify, well, what are successful programs who deal with gang involvement? What are they doing? And we figured out from those programs which of those service objectives meet our program. In other words, can our program meet the following service objectives and make a difference with gang involvement?

So, that's where we're at today. We're identifying additional objectives to develop programs to work with kids who have affiliated with gangs. And, in that process, it'll bring us back into the program evaluation cycle, in terms of eventually designing a program to address the needs of the youth and implementing it. And you can see the program starts all over again.

The cool thing about program evaluation is that's a great way to inform practice. In other words, it tells us whether or not what we're doing is working. With program evaluation, it really ties nicely into evidence-based practice.

Keep in mind, while not trying to bias one toward quantitative research, in most cases when we talk about evidence-based practice, we're talking about quantitative findings that justify such practice. And the reason is this. Many programs that go under evaluation may, for example, be grant funded, or the stakeholders are looking for clear data that says, this works. And from a generalizability prospective, quantitative research fits that model almost more strongly than qualitative research.

That does not discount the need for qualitative data in program evaluation, but you may be challenged with how to objectify that, so you can say that these findings work effectively across the board and a variety of populations.

So, in program evaluation, the implications are twofold. One, for the program and the stakeholders, they obtain data that identifies whether or not their product is working. From a more global perspective, hopefully the program evaluation can be made into a model program and published, so that other programs can examine, hey, here's something successful that's going on. Let's see if we can implement this.

To kind of give you an anecdotal story, I was able to develop an instrument that was published in a counseling journal back in 2007. And what was neat about

Program Evaluation

© 2017-2021 Walden University, LLC 5

that publication is being contacted by psychiatric hospital in Wisconsin that said, hey we saw your article. We saw your instrument. Could we use that at our hospital?

And it provided an opportunity for us to share data and talk about how therapeutic goal attainment might work for adolescents, not just in the psychiatric hospitals I studied, but across the country. So, it's a neat opportunity when you can get a program out there and have an evaluation model that can be generalizable to other people and programs.

Let's talk about some components to look at when you find a program evaluation article and want to know how to implement this into my practice. Naturally, the first thing to look at is is, how does this program evaluation piece tie into what you do? In any particular program, there are probably aspects that will fit your program, and then maybe aspects that don't. So, try and identify what those are. Is it the entire program that you want to bring into your practice, or is it just elements?

Also look at how those elements, then, were isolated. In terms of evaluation, how were the programmatic pieces evaluated both quantitatively or qualitatively? Were the pieces placed together as an entire model? In that case, you really can't separate them. Or were they identified by each element? In which case you can piecemeal it together.

Also, remember to always evaluate the nature of the data. Did they identify statistical significance as a result of using this program, and were the relationships meaningful? In other words, did you evaluate both statistical and practical significance of what was evaluated? So, remember that statistical significance doesn't make it meaningful, and you always have to keep those two aspects separate.

In terms of qualitative data, is what was used transferable? Was it coherent, insightful, and useful? When looking at a program evaluation qualitatively, keep in mind, is it coherent? Is it insightful? Is it useful? And therefore maybe it's transferable to what you're doing.

This ties into the ethical, legal, and multicultural implications to program evaluation. Always keep in mind, is the sample– the participants who engaged in this particular program– a match for the participants that you deal with?

For example, in South Texas at the Garcia Center, most of the youth referred to our services are Latino or Latina youth. So, this is an important consideration when we're providing services. We want to make sure that the family system that we're dealing with is appropriate, and that we're hitting on the cultural issues that mainly Latino and Latina families deal with.

Program Evaluation

© 2017-2021 Walden University, LLC 6

Additionally, you want to consider whether or not these programs are a good fit for the population you serve. As with any program evaluation, the stakeholders probably have a vested interest in making an impact on the community. So, ask yourself, who's in my community? Does this program serve everybody in my community, or does it provide services only for a particular subset?

Another example is that we started providing services specifically geared toward at-risk youth who are female. Why? Because the courts identified a need to begin to address antisocial behavior among female at-risk youth. There we've seen a rise with this particular group. So, we developed specific programs related to adolescent females in anti-social behavior.

Keep in mind that program evaluation can encompass a series of research designs, methodologies, and also employer accounts and skills. It truly puts you in the driver's seat of being able to deliver services, identify community needs, talk with stakeholders, and use multiple methodologies in identifying whether what you're doing is working and is it making an impact.

In this respect, perhaps there's not enough program evaluation research being published in the counseling literature. There isn't a lot of evidence-based practice that we see, and one of the reasons is, is because the primary people who should be doing evidence-based practice research aren't the scholars in the field.

They're the practitioners who are right there on ground zero delivering the services. They're the ones who are most equipped to be doing program evaluation and identifying evidence-based practice. So, hopefully you'll see this as maybe a call to arms, so to say, that as a practitioner, it's important to not just to be delivering services, but to also be thinking about, how can I demonstrate that what I'm doing is working?

Program Evaluation Additional Content Attribution MUSIC: Creative Support Services Los Angeles, CA Dimension Sound Effects Library Newnan, GA Narrator Tracks Music Library Stevens Point, WI Signature Music, Inc

Program Evaluation

© 2017-2021 Walden University, LLC 7

Chesterton, IN Studio Cutz Music Library Carrollton, TX

Program Evaluation

© 2017-2021 Walden University, LLC 1

Program Evaluation Program Transcript [MUSIC PLAYING]

LAURA HADDOCK: Regardless of the setting that any counselor works in there is an ethical responsibility to periodically evaluate the effectiveness of services. Activities that support a counselor's role include program development, program evaluation, and needs assessment. And counselors will conduct planned, systematic evaluations of counseling programs on an ongoing basis to determine the relevance and effectiveness of the program.

The evaluation should include a description of the client competencies or outcomes that were intended– how you intend to collect data and the rationale for a chosen data collection. You want to know whether the therapist accomplished what they set out to accomplish and whether the clients took away what the therapist wanted them to take away. The information that you gather will help the therapist in understanding the strengths and weaknesses of the service delivery and make sure that the counseling program goals are aligned to the individual goals of the client.

Program Evaluation Additional Content Attribution FOOTAGE: WAL_COUN6626F_LauraHaddock.jpg Credit: Laura Haddock MUSIC: SC_Light&Bright06_T32 and/or SC_Business01_T41 Credit: Studio Cutz

Our website has a team of professional writers who can help you write any of your homework. They will write your papers from scratch. We also have a team of editors just to make sure all papers are of HIGH QUALITY & PLAGIARISM FREE. To make an Order you only need to click Ask A Question and we will direct you to our Order Page at WriteEdu. Then fill Our Order Form with all your assignment instructions. Select your deadline and pay for your paper. You will get it few hours before your set deadline.

Do you need help with this question?

Get assignment help from WriteEdu.com Paper Writing Website and forget about your problems.

WriteEdu provides custom & cheap essay writing 100% original, plagiarism free essays, assignments & dissertations.

With an exceptional team of professional academic experts in a wide range of subjects, we can guarantee you an unrivaled quality of custom-written papers.

Chat with us today! We are always waiting to answer all your questions.

Click here to Place your Order Now