Let's look at an analysis done in 2013 by Value Walk. Pay close attention to the green lines showing the difference in costs between combustion engines / electric motor and fuel system / battery system.
There are 2 major cost differences between gas cars and EVs. The first is the Engine/Motor cost. Gas engines cost more to produce. They have a lot more parts and all of these parts must work together. A gas engine can have up to a thousand parts. Below are 2 pictures, the first one shows a disassembled Corvette engine and the second one shows a dissassembled electric motor. You can see that an internal combustion engine has a lot more parts than an electric motor. Manufacturing more parts cost more money as is assembling those numerous parts. This is why an electric motor costs less than an internal combustion engine.
If it is was only for the motor, electric cars would be cheaper than gas engine cars. The tables are turned regarding the fuel system. This includes the fuel tank / battery and also the fuel pump / converter.
On a gas car, the fuel tank is just a big plastic or metal tub containing the fuel. The battery pack in an electric car is composed of a dozens, hundreds or thousands of battery cells, connected together to store enough energy to power the vehicle. As you can see from the pictures below, EV batteries are very complex. They are also made using minerals like nickel and cobalt which can be expensive. Also batteries contain graphite which is very expensive. Because of this, an EV battery is so much more expensive that it offsets the cheaper electric motors and push the price of electric cars quite higher than their gas counterparts.
Note that the price of batteries is coming down every year. Right now, Tesla's battery close is around 150$ per kwh. So we are talking about 15000$ for the battery pack of the Tesla Model S 100D which has a 100 kwh battery pack.