Which component caused the beeps and where is this component located?

The periodic "beep" you hear during a phone conversation would be an indication that it is being taped or recorded. The caller should be getting your permission before recording the call. The component is probably located on the caller's desk, the one who is doing the recording.