Role-playing games like "Dungeons and Dragons" have been around for decades now. The basic concepts of the system are so ingrained in the gaming population that many people have branched out and created their own games. While role-playing games have been created around existing properties and franchises and sold commercially, there is no reason that creative individuals can't make their own game at home. Develop some basic rules and a board, then spend as much time as you want crafting the world and characters to use in your own role-playing game.
Decide what system of basic rules you want to use for governing actions and combat. Since Wizards of the Coast has made the D20 system open and free for anyone to use, this is the system that most gamers choose to adopt. Plus, by basing your rules on the "Dungeons and Dragons" rules, you create a game that most RPG players are familiar with. Of course, feel free to adapt or change rules as you choose to create a system that you feel fits your game world.
Design the world that your game will be played in. Use a generic fantasy world without much detail and just rely on the base rules to get your started if you want. However, detailed worlds are what makes an RPG experience rich and enjoyable. Start by drawing a few maps of the land you want your game to take place in. From there, create a more detailed map and background of a single location to play your first adventure in, such as a single small town.
Print or make a board for use during your adventures. Buy a generic hexagon board surface from a gaming store if you don't want to make your own. Draw your own on cardboard if you are artistic, or design one on the computer and have a print shop print it out for you on laminated plastic, which makes a good game surface. Start with a neutral playing surface with hexagon spaces that you can transform to fit the setting of any of your adventures.
Assemble a large collection of miniatures. Use miniatures from other games if you want to start on a budget, or build a new collection from any of the miniature-producing companies to create a good set of character and monster miniatures to use on the battle map that you've created. Buy some wall sections, trees and other inanimate objects to help transform your basic board into the dungeon, town or forest that you place your players in at any given time.
Invite some friends over and have them roll up some characters based on your basic rule set. Explain that the first adventure is basically a trial while you delve into the world and characters that you've created. See what is fun for the players, what doesn't work well and make notes on the whole adventure. Adapt your rules and world for the next session, and be sure to let everyone know that things are going to change while you refine your new system.
Play some established tabletop RPGs before you start trying to make your own. Most people start with "D&D" and branch out from there. This gives you a taste of what works and what doesn't in established games in the genre.