James Tarkowski's header on the hour mark gave Dyche a winning start to life at Goodison Park and condemned the Gunners to only their second league defeat of the season.
Arsenal's best chance came when Bukayo Saka saw his goalbound effort cleared off the line by Conor Coady as the visitors struggled to threaten Jordan Pickford's goal in the second period.
Defeat left Arsenal five points clear at the top of the table, with title rivals Manchester City playing Tottenham on Sunday, but Arteta thinks his side are still relaxed.
Asked whether his players were feeling any pressure, Arteta said: "No, I don't think so, we've played a lot of games this season like that already.
"But credit to them, this is what they train every single day. You have to defend really well to get your control and earn the right to play. For moments we did it and in other moments we didn't do it well enough to dominate in the game.
"In the last 10 or 15 minutes, particularly after we made the changes, we started to give away a lot of free-kicks and a lot of rash decisions.
"Which is exactly what they want - slow the game down, do everything really long, play long into your line and don't play a lot. We haven't managed that well enough.
"I want the team to know how much I love them. I love them much more now than three hours ago, a week ago, a month ago, three months ago.
"It's very easy to be next to the players when they're winning and performing. These are the moments when I love my players more and my staff more and we're going to stay together.
"This journey is going to be difficult and challenging and there's going to be big stones in the middle and we have to overcome that. Now we have to prepare the week really well to get to Saturday with the right emotional spirit to beat Brentford."
Dyche led Everton to their first win since October, ending a run of 10 matches without a victory under predecessor Frank Lampard, but they remain in the relegation zone.
Dyche said: "Performance is the right word because you can't guarantee the outcome of any game - certainly against the league leaders who are playing so well.
"But I wanted a performance. The players have put a lot of honesty into the week and a lot of respect in the changes we have tried to make, but I said to them, 'It's actions, that's the key'. You could clearly see there was a response."
Dwight McNeil and Tarkowski, two players Dyche managed at previous club Burnley, combined for the winner.
"Of course, I planned it all." Dyche joked. "We believe in our set-piece set-up, it's another valuable weapon for the team to use.
"It was great delivery and a great header and I thought we were causing problems with that today.
"It's a start point. That one (win) doesn't guarantee the next one. That's what I've learned from being a 16-year-old in professional football. It is a start, the way they have taken ownership and pride in what we have tried to do this week - and I value that."