I have seen people asking many times on the forum whether they should use C++ or Python for programming and more often than not they get the response that they should use Python as it is easier to write, ends in a shorter code length and has virtually no performance deficit compared to that of C++. I decided to test this theory and see whether you guys are actually right.
For those people who are too lazy to read I will tell you my conclusion right here: Python does a wonderful job as a scripting language, being easier to code and bearing no performance penalty of significance versus C++. So, if you are going to use Panda3D to write your next game, don’t be afraid to use Python to prototype or even for release.
Of course, this report of mine is not the end-all-be-all answer to this question, it is far from it; however, I think it does a well enough job of answering the question.
I’m posting this here first of all to attempt to clear up this question of Python vs. C++ with some real numbers, but be advised guys that this also a report I did for school and I’m hoping that it will generate some discussion. So please, if you have a comment to make, do make it. If you guys can start some pertinent discussion, that’ll help me on my end a lot. Thanks!
Very cool read, thank you! Something that would of been nice to see would be the p values of your test. For those who don’t know, a p value is a probability that the null hypothesis is true. The lower the p value, the more significant the result. Most published studies aim for p < 0.05, although in psychology, education, and sociology research it can go as high as 0.15 and be acceptable.
You make an excellent point. Rejection of the null hypothesis is most important in a study if one wants to prove their hypothesis over the null hypothesis.
I would certainly have included P-values but there’s just a few problems:
I’m actually still a high school student and the highest level of statistics I’ve been exposed to have been Standard Deviation and z-scores… I do understand what a P-value is and why it is important but I’m still a bit unsure of the proper method of its calculation. That’s not big issue, however. The issue is, is that in the case of this testing, what would one use as a null hypothesis? I’m guessing that would be the opposite of my hypothesis which is that C++ would be faster making the null hypothesis that Python is faster.
But that doesn’t feel right at all, I mean, if my testing had had significant amounts of error and I wasn’t able to draw a conclusion using it, that would make Python faster than C++ by default, which is a ridiculous assumption to make, and so is that C++ and Python would perform equal to one another.
You seem to be knowledgeable in statistics so I’m wondering what would you do in this case?
Technicly it is impossible to prove anything true but we can prove something to be false. So in science and stats we attempt to prove the opposite position to be false. In your case using just basic stats the null hypothesis would claim that there is no difference between python and c++. Using this frame of reference what you are trying to claim is that there is no statistical data to prove you false. In short: You are supporting the null hypothesis instead of the usual trying to disprove it.
Hmm, I actually think that the null hypothesis would be the opposite of what you suggested. See, I stated that public opinion on Python is that it is slower than C++ and just based on that, I think that “Python is significantly slower than C++” would make a more sensible null hypothesis, because, as I understand, it’s something one falls back to if his findings are not significant and what better to fall back to than public opinion which is already widespread?
Now, the problem with this null hypothesis is that it’s tough to determine what “significantly” slower is without really going into all the other factors such as perception on the end-user side of the slowdowns which I sort of went into but didn’t focus on all too much because that’s a whole 'nother aspect of study.
indeed, “The choice of C++ or Python for scripting purposes will have a significant effect upon the resultant performance” might be a good null hypothesis if you’re trying to show that there is little difference.
If you define a significant difference as say 25% performance hit then you will be able to get a concrete P-value. Unfortunately, that’s where my memory of stats runs out so I can’t say exactly how you would calculate it. Maybe looking at the variation of your results and using that to find the chance that the performance hit is greater than 25%? Just a guess though.
