The 50-year-old, who has not been named, was with a group of eight friends who had planned to spend the night in the Priest”s Hole cave on Dove Crag, 4.4 miles (7km) from Glenridding, Cumbria.
Cumbria Police alerted members of Patterdale Mountain Rescue Team at 10pm on Saturday following a 999 call reporting that the man had fallen down the front face of the fell.
A Coastguard helicopter from Prestwick in Scotland assisted in the seven-hour operation, along with Penrith and Kirkby Stephen mountain rescue teams.
Patterdale MRT leader Mike Blakey said: “The helicopter was able to direct the team straight to the man”s location, approximately 50 metres below the bottom of the main crag.
“The man, who had slipped from the ledge, had fallen approximately 150 metres vertically and had sustained fatal injuries.
“This kind of evacuation is always complex as it involves belaying the stretcher down the mountainside and team members literally manhandling the stretcher over each boulder, through dense bracken and over streams.
“We also deployed four team members to the Priest”s Hole to assist the remaining members of the party back to the safety of our base.
“Our thoughts are with the man”s family and friends, including those were staying the night with him. No matter how many times we deal with such incidents they are always tragic and very sad for all concerned.”
He said there had been an increase in visitor numbers to the Priest”s Hole since it featured in a BBC documentary earlier this year, and warned people to be cautious when attempting to reach it.
A man suffered very serious injuries while trying to access the location during the summer, Mr Blakey said.
He added: “”This group of friends were very well equipped and prepared for their adventure. However, as a team, we are seeing more and more people who are attempting to locate the Priest”s Hole in the dark and without the right equipment.
“Indeed, during this rescue we came across three men who had been searching for the cave for a couple of hours.
“We really would like to remind people that the cave is on the front face of a vertical cliff and it is only accessible by one route. It is always best to plan to stay in good weather and to arrive in daylight.
“An Ordnance Survey map and good navigations skills are prerequisites.”