There's an interesting quirk of English here.
"... got to talking ..." is the correct idiom. "got to" here means that you began to do the action. It may be most often used with "talking", but you could say, "I got to thinking about ..." or "We got to eating candy ..." or many other verbs.
"... got to talk ...", curiously, means something entirely different. It means that we were able to or were allowed to. Like, "The kidnappers told us to be quiet, but when they left the room we got to talk." Or less dramatically, "I wasn't allowed to eat candy when I was on the football team, but when the season ended I finally got to eat a chocolate bar."
This works with many verbs. "I have to stand all day at work but when I go home I finally get to sit down." "When I returned from the tropics I finally got to feel cold again." Etc.
I think "got talking" and "got to talking" mean the same thing. I can't think of a difference.