Great question, Kinsey.
I can think of three things that make us sound different from one another, despite speaking the same language: structure, cadence, and jargon.
You’re right that when we try to make people sound different we often only focus on jargon (words, slang, etc. that are specific to a particular community).
Sentence structure is actually a thing of fashion. If you pick up a book pre-1950, most readers will immediately notice something different about the way sentences are constructed. Depending on the time period, you can likely find source material which will give you clues to some appropriate sentence structures. You can achieve this by “priming” or reading things before you start writing that sound like how you want to write.
Cadence is the hardest to achieve in writing, but one of the biggest differences in our speech. Cadence is the speed and rhythm of speech. People in the rural south (USA) speak dramatically slower, on average, than hispanic English speakers in southern California, it’s just a fact. In different regions (and time periods) we also draw out or emphasize different types of words. You can achieve a sense of cadence in your dialogue with punctuation (em dashes, ellipses, italics), and also with the placement and content of your dialogue tags.
This is a big question, and one we could probably talk about for a long time in person! But I hope that these tips get the wheels turning.