Suggestion for new graph to replace histogram(sample program

[Exüberance]

The rating histogram for users' maps has been confusing a lot of people. I suggested that instead of using a map's exact rating, we use it's rounded rating, so the histogram has no ranges, it just becomes a bar graph for each of the 6 possible ratings...

...but then I had a better (in my oppinion, anyways) idea.

How about a cooler-looking, more dynamic graph type? So I made a little flash program that generates what I'm talking about. Feed it a few ratings, and it will make a curve that shows the distribution of ratings by a user. It could probably use a bit of tweaking, but here's the first version:

http://www.box.net/shared/viffgnuuzu (requires Flash Player 8. You should already have Flash Player 10 on your computer, but if nothing happens or it's empty, check to make sure)

Example Screenshots:


What it basically does, is stacks a bunch of bell curves ontop of each other and scales it. If you are interested, I'll post the source code in this thread.

Comments? Suggestions? Is it better than the current histogram? Would you prefer this or a more simple bar graph?

[Arachnid]

You're right, this is a really cool idea.

There's one problem with implementing it: It's impractical to sum up all ratings every time the graph needs to be generated. This can be solved by quantizing into buckets and updating them when they change, though - which is what NUMA already does for the histogram, this would just have more buckets (somewhere between 20 and 100 is probably reasonable).

Edit: Want to add this as an issue on uservoice? You can link to this topic in the description.

[Exüberance]

Okay! That will be easy to modify. All I have to do is weight the buckets according to their size. Just so long as there is enough buckets to make the graph accurate instead of just a few lumps. (between 20 and 100 would probably look just fine. Most people dont even have 100 maps) I posted this suggestion under the "historgram only goes to 4" comments in the user voice already, but I can make another one if that would be more appropriate.

I might as well post the source code here: (forgive my complete lack of comments, I'll add them later. I have a bad habbit of not commenting programs if they are very short.

Code: Select all

//This is the code for the button which does all the work

on(press) {
	var bell:Function = function(x:Number) {
		return(Math.exp(-1*x*x));
	}

	var ratings:Array = new Array();
	ratings = _root.ratingInput.split(",");
	
	var fail:Boolean = false;
	_root.err._visible = false;
	
	_root.rateGraph.clear();
	
	for (var i:Number = 0; i < ratings.length; i++) {
		ratings[i] = parseFloat(ratings[i]);
		if (isNaN(ratings[i])) {
			fail = true;
			_root.err._visible = true;
			break;
		}
	}
	
	if (ratings.length == 0) {
		fail = true;
		_root.err._visible = true;
		break;
	}
	
	if (fail == false) {
		_root.rateGraph.lineStyle(1,0x0000FF);
		_root.rateGraph.beginFill(0x0080FF);
		
		var ylist:Array = new Array(125);
		var max:Number = 0;
		
		for (var i:Number = 0; i <= 125; i++) {
			ylist[i] = 0;
			for (var n:Number = 0; n < ratings.length; n++) {
				ylist[i] += bell((2.5 * ratings[n])-(i/10));
			}
			if (ylist[i] > max) {
				max = ylist[i];
			}
		}
		max = -32 / max;
		
		_root.rateGraph.moveTo(0,0);
		for (var i:Number = 0; i <= 125; i++) {
			_root.rateGraph.lineTo(i,max * ylist[i]);
		}
		_root.rateGraph.lineTo(125,0);
		_root.rateGraph.lineTo(0,0);
		_root.rateGraph.endFill();
	}
}

The only other code is 3 lines in the err instance (the "Error Generating Graph" text) which makes it invisible by default.

Instances:
err - the error text
rateGraph - the movie clip where the actual graph is drawn to
<no instance name> (the button) - contains the code above. Has no instance name since nothing else references it
ratingInput - the ratings you input

[Arachnid]

I just saw your comments on the existing bug. That works fine.

The buckets all represent an equal area - the value is the number of maps that fall within that interval. My main concern is that this graph will be ugly for some users, particularly those with a small number of maps - if we have 20 buckets, and the user has 10 maps, it's going to be a very spiky graph.

And I appreciate your prototype, but any implementation will almost certainly use the Google Charts API - I don't want to require flash to view histograms. :)

[Exüberance]

I just saw your comments on the existing bug. That works fine.

The buckets all represent an equal area - the value is the number of maps that fall within that interval. My main concern is that this graph will be ugly for some users, particularly those with a small number of maps - if we have 20 buckets, and the user has 10 maps, it's going to be a very spiky graph.

And I appreciate your prototype, but any implementation will almost certainly use the Google Charts API - I don't want to require flash to view histograms. :)

Ah, I see. I misunderstood the bucket idea (I thought you meant you take X maps, average the rating, and count it as a single map). This way works much better. Currently, the system uses 125 buckets (sort of, not really) for each pixel. Using instead, say 50 buckets would work just as well.

I guess the implementation would instead of iterating through 125 pixels, would go through maybe 50 (2.5 pixels each). And it would go through each bucket, checking the number of maps in it. It would then add that value into the corresponding bucket of another initially zero graph, and a fraction of that number to the surrounding buckets (decreasing fraction the farter away from the initial bucket you go. (I used the bell curve (e^(x^2)) in my implementation). Then you scale the values so the highest value touches the top of the graph.

I'm not sure if you can write code or anything with the Google Charts (I didn't even know it existed till you mentioned it), but I image you could probably find something that does that or something similiar.

[Arachnid]

I've been giving it some thought and come to much the same conclusion as you.

If you want to help, what I could really use is this: An algorithm that takes as input an array of n buckets consisting of equal regions between 0 and 5, with the number of maps rating that value in each, and generates a (larger) array of a specified size describing the points of the graph to generate.

For example, say we decide to use 50 buckets (0.0-0.1, 0.1.-0.2, ..., 4.9-5.0) and we're generating a 150 pixel graph, it would take a 50 element array and return a 150 element array, smoothed by your algorithm.

I have some misgivings about the fact that the smoothing reduces the accuracy, but it's cool enough that I think maybe we can let that slip. ;)

[Exüberance]

Alright, I'll be happy to help! I can use "steeper" graphs for more accuracy if you like, but I'll keep the current algorithm for now.

[epigone]

This sounds like a really good idea, and I'm glad Arachnid is for it (plus he noticed the slight impracticality issue which I also noticed, but didn't really know how to explain). I hope that something similar to this gets implemented. If you haven't yet, definitely add this to uservoice.

[Exüberance]

Okie dokie, the program is done. It looks like of bumpy for people with very few maps, but that can be fixed. I was thinking, maybe we could make the "steepness" or "sharpness" of the curve depend on how many maps a person has if they have fewer than X maps. Otherwise, use a constant value. That way a person with 5 maps would get more of a blob than a wave function. Then again, maybe it's actully better this way *shrug*

Right now, it gives you a rediculous amount of decimals. I can make it round to 1 decimal or to the nearest integer if it would be better.

Anyways, here's the program. (I did it in Flash again)
http://www.box.net/shared/bcooxbf3rk

And the source code:

Code: Select all

on (press) {
	
	var bell:Function = function(x:Number) { //the bell curve function
		return(Math.exp(-1*x*x));
	}
	
	_root.err = ""; //clear error text
	var fail:Boolean = false; //incase something goes wrong. No easy way to break out of a event in Flash :(
	
	var a:Array = Array(); //easy and inefficient! Woo!
	a = _root.inputArray.split(","); //nice little feature that converts text into an array
	
	var b:Array = Array(_root.graphWidth); //the output array that will contain the heigh of each pixel in the graph. Creative variable names, I know.
	
	for (var i:Number = 0; i < a.length; i++) {
		a[i] = parseInt(a[i]); //converts string to real number. Flash lets you chance variable types in arrays. Isn't it nice ^_^
		if (isNaN(a[i])) { //oh no! Something went horribly wrong!
			fail = true;
			err += "Error: Unable to parse input. Non-numeric entries detected.\n"; //better than a segfault!
		}
	}
	var k:Number = 0; //more descriptive variable names! WoOoOoO! For now this variable records the maximum height, but that will change later
	
	//the following loop does the initial pass through the graph, establishing the ratios between the pixels
	for (var i:Number = 0; i < _root.graphWidth; i++) {
		b[i] = 0;
		for (var j:Number = 0; j < a.length; j++) {
			b[i] += a[j] * bell((i - (j * _root.graphWidth / a.length))/10); //currently, the algorithm uses a bell curve where 10 pixels is 1 unit. Changing the 10 to a smaller number will increase the accuracy, while making the curve more sudden
		}
		
		if (b[i] > k) { //if b[i]'s dad can beat up k's dad
			k = b[i]; //we have a new champion!
		}
	}
	
	k = _root.graphHeight / k; //now k is the constant we multiply everything by to scale it to fit the graph
	
	//this second pass scales the graph such that the highest value is the height of the graph
	for (var i:Number = 0; i < _root.graphWidth; i++) {
		b[i] *= -1 * k;
	}
	
	if (a.length == 0) {
		fail = true;
		err += "Error: No input detected!";
	}
	
	if (fail == false) { //now print it to the textbox
		_root.outputArray = b.toString();
	}
	
	//aaaaaand we're done! Coffee break!
	//oh wait, the sample graph!
	
	_root.graph.clear();
	_root.graph.lineStyle(1,0x0000FF);
	_root.graph.moveTo(0,b[0]);
	for (var i:Number = 0; i < b.length; i++) {
		_root.graph.lineTo(i,b[i]);
	}
	
	_root.graph.lineStyle(1,0);
	_root.graph.moveTo(0,0);
	_root.graph.lineTo(0,-1 * _root.graphHeight);
	_root.graph.moveTo(0,0);
	_root.graph.lineTo(_root.graphWidth, 0);
	
	trace(b.length);
	//ok, NOW we're done!
}

H'uh. The code tags still use word wrap. That's stupid.

EDIT: After testing it out a bit, I think it might be better to make the curves turn a faster so they don't bleed into the surrounding ratings as much. Maybe double the rate? To do this, I would change the...

Code: Select all

b[i] += a[j] * bell((i - (j * _root.graphWidth / a.length))/10); //currently, the algorithm uses a bell curve where 10 pixels is 1 unit. Changing the 10 to a smaller number will increase the accuracy, while making the curve more sudden

...line to have a 5 instead of a 10

[Yoshimo]

Sounds pretty cool, could you just internet it? I just installed flash 10, but it only works on safari; I can't figure out how to do it with .swfs... >_>

Edit: Your second program works. Not sure what it does though, it just gives me a bunch of waves. Should I add more variables?

[Exüberance]

Sounds pretty cool, could you just internet it? I just installed flash 10, but it only works on safari; I can't figure out how to do it with .swfs... >_>

Edit: Your second program works. Not sure what it does though, it just gives me a bunch of waves. Should I add more variables?

yeah, the second one doesn't look that good for big partitions (fewer numbers). In the first one you input ranks, in the second one you input the number of maps in each partition.

If you want to view it in your browser, just choose Safari / Firefox / Konqueror / Opera / Internet Failsplorer when it asks what program you want to open it with. (If you've downloaded it, right click and choose Open With assuming Windows. No idea what it is on a Mac. Maybe the same?)

[Arachnid]

Very nice. A pity the algorithm is O(mn), but hopefully that won't matter for the number of buckets and graph size we're dealing with.

Also, I can't help noticing that all the points your sample app output are negative. :)

[Nexx]

This is very cool and I support this.

However, I must admit I feel rather sore that my simpler and more obvious fix was treated poorly and then ignored, but this much more complicated suggestion is immediately latched onto. ;____; Don't I make good suggestions?

[Exüberance]

Very nice. A pity the algorithm is O(mn), but hopefully that won't matter for the number of buckets and graph size we're dealing with.

Also, I can't help noticing that all the points your sample app output are negative. :)

Technically the algorithm will be O(n) time efficiency because we willl be using a constant number of partitions/buckets, so one of those numbers will be a constant. EDIT: I suppose graph size is constant too so techinacally it would be O(1), but now I see what you mean. Yeah, it takes a little bit of time, but it's a small amount of time for me, and Flash wasn't really built for efficiency. Sometimes I wish they would add the time and space efficiency of operations into the documentation.


I have updated both files with an option to change the width of each curve. (example attached). I also added an option to show output y-coordinates as positive. They were negatives before becuse Y increases as you move down the screen, so to have the height of the curve increase, Y had to decrease, so it was negative with respect to the origin of the graph.

[Arachnid]

Very nice. A pity the algorithm is O(mn), but hopefully that won't matter for the number of buckets and graph size we're dealing with.

Also, I can't help noticing that all the points your sample app output are negative. :)

Technically the algorithm will be O(n) time efficiency because we willl be using a constant number of partitions/buckets, so one of those numbers will be a constant.

By that reasoning, it's O(1), because we'll use a fixed graph size, too. But we're actually considering the algorithm in the abstract, and how the number of buckets and graph size affects the running time.

I have updated both files with an option to change the width of each curve. (example attached). I also added an option to show output y-coordinates as positive. They were negatives before becuse Y increases as you move down the screen, so to have the height of the curve increase, Y had to decrease, so it was negative with respect to the origin of the graph.

Depends on your coordinate system. :)

[Yoshimo]

When I open the second one, there's this weird black box-ish thing in it. See Attached image for image. Also, it goes away with the mouse over the red button.

P.S. But the 'blender' to 1 then input this string: 5,3,5,0,2,3,0,5,5,3,5,0,2,5,1,5,5,3,5,4,2,3,1,5,5,3,5,0,0,0,0,2,3,0,5,5,3,5,0,2,5,1,5,5,3,5,4,2,3,1,5,0,0,0,0,0,0,0,5,3,5,0,2,3,0,5,5,3,5,0,2,5,1,5,5,3,5,4,2,3,1,5,5,3,5,0,2,3,0,5,5,3,5,0,2,5,1,5,5,3,5,4,2,3,1,5

[Exüberance]

When I open the second one, there's this weird black box-ish thing in it. See Attached image for image. Also, it goes away with the mouse over the red button.

P.S. But the 'blender' to 1 then input this string: 5,3,5,0,2,3,0,5,5,3,5,0,2,5,1,5,5,3,5,4,2,3,1,5,5,3,5,0,0,0,0,2,3,0,5,5,3,5,0,2,5,1,5,5,3,5,4,2,3,1,5,0,0,0,0,0,0,0,5,3,5,0,2,3,0,5,5,3,5,0,2,5,1,5,5,3,5,4,2,3,1,5,5,3,5,0,2,3,0,5,5,3,5,0,2,5,1,5,5,3,5,4,2,3,1,5

o.O

that's... interesting? I don't recall using orange anywhere in that program. Make sure Flashplayer is updated. I can't get that glitch on my computer.

Also, just so you know, the second program takes the number of maps in each partition of the rating system (if you input 50 values, the partitions are 0 - 0.1, 0.1 - 0.2, etc. If you enter 5 values, it's 0 - 1, 1 - 2, 2 - 3, 3 - 4, 4 - 5), not map ratings. If you want to see what it would look like with maps rated as you typed in, use the first one.

[Yoshimo]

I realize. I just thought it looked cool. Also, I will try.

Edit: I opened it with safari and it fixed itself. I bookmarked it. ; D

[Arachnid]

One slight problem with your current algorithm - the graph's shape actually depends on the width of the graph. Try entering a set of figures, then changing the width of the graph.

Easily fixed, though: Just normalize the x axis to be from 0.0 to 1.0, regardless of the actual width, and express the decay in terms of that.

[Arachnid]

This is now live. With one slight optimization: I precalculate a single canonical bell curve at startup, so I don't have to call the expensive math.exp function thousands of times for each graph. :)

[Erik-Player]

This is amazing guys. Really.

[Radium]

Awesome. Nice job!

[Mae]

Much better =D

[Exüberance]

This is now live. With one slight optimization: I precalculate a single canonical bell curve at startup, so I don't have to call the expensive math.exp function thousands of times for each graph. :)

:)

Oh yeah. That makes a lot more sense! Anyways, I just noticed this now. Awesome. What are you using to generate the graphs, th Google thing you were talking about before?
Hah, I'm doing pretty well with ratings. It peaks farther along the x axis then I thought it would. I think the size of the curve is perfect; it looks really good.

[Arachnid]

This is now live. With one slight optimization: I precalculate a single canonical bell curve at startup, so I don't have to call the expensive math.exp function thousands of times for each graph. :)

:)

Oh yeah. That makes a lot more sense! Anyways, I just noticed this now. Awesome. What are you using to generate the graphs, th Google thing you were talking about before?
Hah, I'm doing pretty well with ratings. It peaks farther along the x axis then I thought it would. I think the size of the curve is perfect; it looks really good.

Yup, I'm using the Google Chart server. The curve is generated in Python on the server-side, then encoded in a chartserver URL. Just check the URL of the image to see.