One of the apps on your phone is unlike all the others: the web browser. There are many reasons it is different but here is the most fundamental one – you tell it what to connect to! Pretty much every other app connects to a specific set of servers that the app’s creators control. So the browser gives you as enduser a fundamentally higher level of control than most other apps.
What makes this possible is the existence of the HTTP protocol. So what we need in order to shift more control back into the hands of users are more protocols and apps that implement them. An interesting one would be some kind of update of IRC. There are lots of existing IRC clients you can run on your phone and they too like a browser let you connect to different servers. But IRC dates back to 1988 and has no provisions for the many capabilities of a smartphone.
Here is a small outline for what a new chat protocol might look like. When you first connect to a server (or a “room”) the server tells you what information it needs from you to participate. This could be as little as, well, nothing really for completely anonymous chat (could even support TOR for routing). Or it could require you to share some kind of identity, possibly your location and maybe even your device fingerprint. This initial permission step would allow many different types of service to be implemented on the same protocol, including ones similar to GroupMe or YikYak.
Once you are connected, the server streams updates to the “room” to you. These would include new message posted as well as (delta) updates to existing messages. Each message would consists of some kind of JSON structure including the message itself but also meta data, for instance to support upvote and downvote counts, but also attachments such as pictures. One possible candidate for this step is the work Jeremy Miller is doing with Telehash.
The other missing ingredient then is discoverability for servers (rooms) to join. This portion could happen via the web. Ideally there would be a single URL scheme and all apps that support the protocol can register for that (this would require iOS and Android to let different developers submit apps that register the same scheme – maybe if the scheme is owned by a foundation they would allow this). You could then discover a room on the web or have one sent to you in any other way (email, SMS, etc) and have your personal favorite and trusted app take you there.
It easy to look at the current state of centralization and walled gardens and despair. But if we take a longer historic view we see that we have gone through these kind of waves before. We had highly decentralized dial-up services. Then we got the period of big centralized dial up like AOL and Compuserve. They were supplanted by the decentralized and protocol based web which in turn has been partially supplanted by closed and stove piped apps. But this does not have to be the end state if we don’t let it. An updated version of IRC seems like a great starting point to me.