To answer that question, I'll start with a brief description of a typical roleplaying game:
"A group of people sits around a table, with formalized descriptions of imaginary characters on pieces of paper in front of them. There are normally dice on the table, and sometimes even models representing the imaginary characters. Everyone starts talking at once, usually loudly; rolling dice and ignoring results, and scribbling notes on their 'character sheets.' If an outsider were able to discern what was going on in this hubbub, he would find that it boiled down to a protracted question and answer session between the 'players' and 'referee.' It would not be too much of an imagination to say that the entire roleplaying hobby is a series of subtle and complex elaborations of the formula:
"Referee: 'What do you do now?'
"Player: 'I do such-and-such.'"3
But this hardly answers the question. A roleplaying game is, in essence, the construction of a story (using the term loosely) by a group of people imagining the happenings in a world, restricted only by the consensual reality of that world, and assisted to some degree by formalized rules of play. An RPG is a strangely difficult concept to explain, though I have found it an exceptionally easy concept to demonstrate. On the one hand, everybody already knows, because they've spent their lives doing it--pretending. But on the other hand, it is a strange, new concept--rules, jargon, worlds of another's invention. To further complicate matters, there is much disagreement between roleplayers on just which characteristics are fundamental, and which define sub-genres. The best description I've seen for encompassing all RPGs, and still separating them from storytelling and acting and other forms of entertainment that most roleplayers would intuitively define outside of roleplaying is a "participative, collaborative, extemporary, storytelling amusement."4 Continuing with Brett Evill's definition of an RPG:
"The facts that everyone participates (no-one is just audience), that the storytelling is collaborative (not directed by one person), and that it is extemporary (the story emerges during play, it is not determined in advance) make RPG storytelling different from writing books or plays. The fact that it is a storytelling game, not a storyhearing game, makes it different [from] reading a book or watching a movie."5
Evill makes fairly clear how an RPG is unlike reading, but also brings up the parallel with storytelling. There are certainly elements of acting and game in an RPG, too. So how does it differ from these?
First of all, an RPG is not a game by most definitions. "A game is a form of art in which participants, termed players, make decisions in order to manage resources through game tokens in the pursuit of a goal."6 An RPG almost fits this definition, but it lacks an inbuilt goal. Like SimCity (itself not strictly a game),7 there is no inherent goal to an RPG. Many provide settings where certain goals are likely to be adopted by the players, but they are not the only goals, nor are they necessary goals.
An RPG is not quite acting, though it shares a lot of characteristics. In particular, improv theater and some of the nearly rules-less RPGs come very close to each other. But theater has performers and audience, and while that distinction is occasionally bent, it is essentially never broken. RPGs have no audience, in the sense that there is no intended place for someone who is passively observing without participating. Or, you can consider all the participants to be simultaneously audience members, appreciating each other's performances.
An RPG is not storytelling, though it is very close. Again, some of the more unusual RPGs are not much different from a group storytelling session. But always, the primary goal of storytelling is to tell a worthwhile story. In an RPG, other things often take precedence over the storyline, such as enjoyment of the participants, advancing individual characters, or obeying the rules. And even at the extreme, the players in an RPG have more input than the audience of a storyteller. Yet if you go to the other extreme, where the boundary between referee and player has been completely eliminated and the roles blurred, everyone has more authority than any collaborative storyteller, since there is essentially no time when they do not have input.
So, considering what they are not, what are they? There are certain commonalities to essentially all RPGs. Every RPG involves characters, and the majority of them match one character to one player. These characters are much like those of a play or novel in conception, but the game-like elements come in during their construction. In order to maintain a sense of balance or "fairness,"8 there are rules for quantifying the characters in terms of the game reality. Most RPGs allow the players to allocate a pool (or several pools) of points towards several categories of traits, including "attributes," which measure raw ability, like strength or intelligence; skills; personality traits, such as "brave," or "patient;" and whatever special powers the game reality affords, such as spell casting or psychic abilities or Aston Martins with hidden machine guns and an ejector seat. There is actually quite a bit of variety in the details of these systems, but all have in common the same goal: to quantify the player's conception of the character, thus eliminating the biggest problem of childhood roleplaying--"I shot you!" "No you didn't!"
Every RPG has mechanics of some sort for resolving what happens in the game reality. The variety in the details, however, is huge. Some strive to be an extremely realistic simulation of reality or a particular genre (Kung Fu films or comic-book superheroes). Others emphasize speed or simplicity of play. Most at least make a stab at both of these goals, to varying degrees. Most use dice for a random element, to simulate all the many little things that influence an outcome in real life, but aren't quantified by the game. A few instead use playing cards or tarot cards, at least one game uses poker chits and another marbles, and a very few have no such random element. The specific details of how the mechanics work vary widely within this basic parameter, involving various ways of rating traits and rolling dice of anywhere from 4 to 100 sides and/or drawing and playing cards, or marbles, or perhaps something else.
As mentioned above, there is generally a referee, usually referred to as the "Gamemaster."9 This person is responsible for everything that the individual players aren't. This includes creating and describing the settings, roleplaying other characters encountered (NPCs10), adjudicating the rules, maintaining the plot, and so on. The gamemaster is analogous to the narrator and editor of a novel, where the players are playing the main characters. However, some RPGs play with this dichotomy, changing the balance of responsibility between gamemaster and player. Two common changes are to shift more of the burden away from the gamemaster, or to blur the distinction between the two. In the former case, the rules are constructed in such a way that they center around the players, letting them do all the die-rolling and most of the adjudication of results. The latter involves sharing world-creation and narration amongst all of the participants, either through taking turns as the gamemaster, or through divvying up the responsibilities in some fashion.
One other interesting commonality shows up looking at RPGs. They nearly all take place in a fantasy world, and they all revolve around the characters being exceptional individuals. A very few published RPGs make it possible to play nominally average individuals, but none make them the focus of the setting. At least the background or situation of the characters is unusual.11 And the closest any of the settings get to the real world are some modern military games, which at least attempt to be an accurate simulation of elite military squadrons, such as the Green Berets and Rangers. Assuming the genre simulation is accurate, such a setting is still essentially a wish-fulfillment fantasy world for the players. Most RPG settings are much more overtly fantastic, such as Middle Earth (from The Lord of the Rings) or Victorian England with faeries and trolls, or the Star Wars universe.
Beyond these four general commonalities, not much can be said of roleplaying games as a whole. They run the gamut from extremely serious through satirical to silly. Genres include action-adventure, spy, space opera, swords & sorcery, modern fantasy, historical fantasy, historical, horror, comic book superhero, cartoon, Anime, and nearly anything else you can think of. Some games have extremely detailed settings, while some have none.12 Some have very concrete rules, while others are little more than guidelines.