In 31 years as head coach at Seton Hall from 1973-2003, Sheppard recorded 28 winning seasons and 27 postseason berths, highlighted by 15 BIG EAST Tournament appearances. He was named the BIG EAST Coach of the Year three times (1985, 1987, and 1989), and guided the Pirates to the conference tournament title in 1987. Sheppard's squads reached the NCAA Tournament 12 times while he led Seton Hall to appearances in the College World Series in 1974 and 1975.
Under Sheppard's tutelage, the Pirates won 20 or more games 30 times, 30 or more games 22 times, and 40 or more games five times. He holds a career record of 998-540-11 and ranks in the top 75 on the list of NCAA's all-time winningest coaches (by victories).
During his tenure, more than 80 Seton Hall players have gone on to sign professional contracts, with 30 of them moving on to play in the major leagues. He coached John Valentin to a fifth round draft pick plus eight first round draft picks that consisted of such stars as Craig Biggio, Rick Cerone, Jason Grilli, Matt Morris, Mo Vaughn, and John Valentin.
In 1989, Sheppard was honored as a "Giant Steps" Award winner by the Northeastern University Center for the Study of Sports Society. In addition to being inducted into the Seton Hall University Hall of Fame in 1996, and having his jersey number 17 retired on April 25, 2004, he earned the McQuaid Medal in 1990 which is an award given for 20 years of academic and athletic service to the Seton Hall community. Finally, Sheppard was inducted into the American Baseball Coaches Association Hall of Fame on January 7, 2011.
Though currently retired from coaching baseball, Sheppard remains active in the Seton Hall community as the Head Coach Emeritus, while also continuing his duties as the Chairman of Essex County American Legion Baseball, a post held since 1983, and the Commissioner of Atlantic Baseball Confederation Collegiate League
Sheppard and his wife, Phyllis, reside in West Orange, N.J. All five of their children, Mike Jr., Susan, Kathleen, John, and Rob are Seton Hall graduates with Sheppard's three sons and son-in-law all being Seton Hall baseball standouts. His oldest son, Mike is currently the head coach at Seton Hall Prep while his youngest son Rob is entering his ninth season at the helm of Seton Hall University's baseball team. His son John is currently a teacher and head baseball coach at Morristown-Beard High School, while his son-in-law Ed Blankmeyer, a player and assistant coach at Seton Hall, enters his 17th season as head coach of St. John's.