Note the altered title: Baby Let Me Can I Take You Home.

Single on His Masters Voice by New Zealand band from Christchurch, also working in Australia 1966-68. They changed their name to simply The Castaways, logically, after Peter Nelson (b. Trebilcock) left for a solo career during their time in Australia. See Skye Boat Song (#6 Sydney).

Further reading: Castaways history at Bruce Sergent's NZ Music website.

Single on Keetch, May 1964, produced by Bert Berns (aka Bert Russell).

Used the backing track from the Hoagy Lands version of January 1964 (below) although it is less busy, and closer to The Animals' version.

The reliable Fred Clemens, for one, (commenting at 45Cat.com) says that The Mustangs were in fact Bert Berns and Wes Farrell.

The Animals' first single, on Columbia, March 1964, arranged by Animals keyboardist Alan Price, based on arrangements by Bert Berns. In the US the B-side I'm Gonna Take You Back To Walker became the A-side.

Chronology, and some speculation: The Mustangs' version is often cited as the source of The Animals record (which sounds right from listening to the two tracks) but The Animals' single was released a couple of months before The Mustangs' single. Animals producer Micky Most brought Baby Let Me Take You Home back from a 1963 visit to New York where he consulted with Bert Berns. It is plausible that Berns acquainted him either with an unreleased Mustangs recording (released later in the wake of the Animals release), or with a demo recording, which possibly amounted to the same thing, considering that the Mustangs were Bert Berns and his collaborator Wes Farrell.

The Bert Berns-produced singles by Hoagy Lands (below) and by The Mustangs have some key elements in common with the Animals version, notably the distinctive guitar introduction and an important melody change. The Mustangs single apparently uses the same backing track from Berns's production of Hoagy Lands, but the feel is closer to The Animals, and thus closer to trends in beat music that were emerging from Britain at the time. Lands's record, with vocals reminiscent of Sam Cooke, sounds more old-school, and his 30-second sung introduction is dropped on the Mustang's record. The Animals drop it too, along with embellishments by a female vocal group, and they pare it down further by replacing a vocal line at the end of each verse with an intstrumental break. The Mustangs and The Animals also share the fresh title, Baby Let Me Take You Home.

Certainly, in versions of the song from January-May 1964 by Hoagy Lands, The Animals and The Mustangs, the main melody has been tweaked by Berns enough to make it stand apart from the numerous earlier versions, right up to Bob Dylan's (1962). In fact, a casual listener familiar only with, say, the Animals version might not recognise Baby Let Me Take You Home in a song such as Don't Tear My Clothes from 1935.

Selected references, further reading:
1. Joel Selvin, Here Comes the Night: The Dark Soul of Bert Berns and the Dirty Business of Rhythm and Blues (2014):

The Animals' version of Berns and Farrell's Baby Let Me Take You Home - an arrangement almost identical to the demo Berns and Farrell recorded that served as the basis for their Mustangs record, not yet released - made a respectable showing at number twenty-one on the UK. charts... [link]

2. The Animals by Stevie King at British Blues Archive

History tells us that the Animals single was released in March and the Mustangs one in May, but whether this was Berns issuing a "cover version" of the song for the US market after the Animals UK success, or whether Most had originally brought a Mustangs demo to the UK, has been lost in the mists of time. [link]

Single on Atlantic, produced by Bert Berns (aka Bert Russell). The song edges towards the well-known version by The Animals, although it still had some changes to go through before then.

At around 0:30, after a slow introduction, , you can hear the distinctive guitar line that kicked of The Animals' Baby Let Me Take You Home, released a couple of months later. The melody now takes a downward turn at the end of the first line, setting it apart from previous variations on the traditional song and making it recognisable to those familiar with The Animals' version.

A current Beatles hit is echoed by the title Baby Let Me Hold Your Hand. Could I Wanna Hold Your Hand, taking off on the US charts when this single came out in January 1964, have reminded Berns of Professor Longhair's 1957 record Baby Let Me Hold Your Hand (see below)?

On his first album, Bob Dylan.

In  a spoken introduction to the song, Dylan says:

I first heard this from Ric von Schmidt. He lives in Cambridge. Ric is a blues guitarplayer. I met him one day on the green pastures of the Harvard University.

See commentary at Bob Dylan's Musical Roots which casts doubts on the attribution of the song to Rev. Gary Davis.

Also recorded in the 60s by Rev. Gary Davis and by Dave van Ronk in 1964.

On Folkways album New Orleans Street Singer.

Writing about The Animals' 1964 recording of Baby Let Me Take You Home, Joel Selvin notes that Eric Burdon recognized the song from blind New Orleans blues singer Snooks Eaglin's Mama Don't You Tear My Clothes, another variant of the song... [in Here Comes the Night: The Dark Soul of Bert Berns and the Dirty Business of Rhythm and Blues (2014)]

Single on Ebb, B-side of Looka, No Hair.

The writer credited on the label, R. Byrd, is Professor Longhair, the New Orleans singer-songriter-pianist born Henry Roeland Byrd, also known as Roy.

In 1964, Bert Berns would produce and co-write another version called Baby Let Me Hold Your Hand, sung by Hoagy Lands. Berns also used the pseudonym Russell Byrd, but that's just a potentially misleading coincidence.*

* I was misled, for one. Thanks to Joop Jansen for the correction.

Recording on ARC label by Fulton Allen.

Recorded a few months earlier by Walter Coleman whose version was issued later. 

See Bob Dylan's Musical Roots (which questions the attribution of the song to Rev Gary Davis). Arnold Rypens' The Originals discusses the song's origins and lists some other versions and variations.

78 rpm record on Vocalion, also released on Okeh, recorded with the participation of Big Bill Broonzy (guitar) [¿and harmonicist Jazz Gillam?]. 

78 rpm record on Vocalion, recorded in Memphis by husband-and-wife duo Wilbur "Kansas Joe" McCoy (1905-1950) and Lizzie McCoy, née Douglas (1997-1973), aka Memphis Minnie.

An early prototype of what became the Bob Dylan-Animals song. Memphis Minnie sings Do anything in this world I can... where Bob Dylan and The Animals sing I'll do anything in this Godallmighty world...

Song probably first recorded in 1930, variously known as Can I Do It For You, Don't Tear My Clothes, Mama Let Me Lay It On You, Baby Let Me Follow You Down, Mama Don't You Tear My Clothes, Baby Let Me Hold Your Hand, Baby Let Me Take You Home.