Default joomla installation generates url with index.php file in it. Now if you want to remove index.php from the url then you can use mod rewrite with .htaccess or you can use 3rd party SEF extension to remove index.php from the url.
To remove index.php from the url, go to your site root folder. You will see a file name htaccess.txt. Rename it to .htaccess and upload the file to your root folder. Now go to System->Global Configuration. Now you will see SEO Settings like below image:
Now click on Yes to Use URL rewriting. Then your url will be like http://www.yoursite.com/about-joomla-site.
If you want to make url to http://www.yoursite.com/about-joomla-site.html then you need to click on Yes to Adds Suffix to URL.
Then press Save button at top left corner. That will save your global configuration. Sometimes it shows an message like “An Error has occurred! Unable to open configuration.php file to write!”. In that case go to ftp client, login to your site and give write permission to configuration.php file. You can give 755 or 644 to configuration.php file. Then error will go away. Now browse the site and you will see you got your desire url, just the way you wanted.