Witnesses said they saw a man struggling to keep his head above water around 11:30 a.m. near the Longfellow Bridge. But no one was able to save him.
Police pulled his body from the water just after 12:45 p.m., Massachusetts State Police said.
Dive teams and searchers from multiple agencies took part in the search for the man.
The man's name was not released as investigators notify his next of kin.
The Office of the Chief Medical Examiner will perform an autopsy. Police are investigating what led to him ending up in the water and where he went in.
Click here to follow Daily Voice Middlesex and receive free news updates.