Просмотр кода страницы Development:Interactive paint tool

GSoC 2009 Project Proposal for Moodle (-penultimate draft)

Submitted By:

[http://www.riteshmehta.741.com/ Ritesh Mehta],

Third Year Undergraduate Student,

[http://www.cusat.ac.in/ CUSAT],

India

== '''Proposal''' ==
Here I propose an effective and much needed tool to augment online teaching in the form of interactive drawing, painting, editing images that could be shared among users and the final work be saved to the server or user machine.
Such a tool could be used by an instructor to instruct a group of students in courses like Physics, Geometry, etc that can make use of diagrams for better flow of ideas.

Another viewpoint is in case of kids using OLPC XO laptops which actually got it listed as one of the Moodle project ideas for GSoC 2009 (WYPIWYG extension to WYSIWYG editor) and had been a topic of vivid discussion on the Moodle Development Forum for quite a while now.
There have been detailed discussions about it on the following Forum threads (in chronological order)

Refer:
'''Forum'''
[http://moodle.org/mod/forum/discuss.php?d=110902]
[http://moodle.org/mod/forum/discuss.php?d=119477]
[http://moodle.org/mod/forum/discuss.php?d=119670]

'''Tracker Issue''':[http://tracker.moodle.org/browse/MDL-18651]

An advanced tool with expanded capabilities can also be used for Professional work/Painting courses.

The objective of the proposed tool is not only to let the users draw and save their work but to work out efficient algorithms to share the drawing area in real-time, so that multiple users can draw on the board at the same time and everyone gets the same view at all times. This real-time aspect of interaction thus makes networking and synchronization an important area of focus.

Such a feature can prove to be the genesis for a new kind of virtual class rooms in which we can combine real-time chat rooms with this tool to give online learning look more similar to face-to-face learning.

== '''Proposed Plan of Development''' ==

=== Design Objectives ===
Following the discussions on the forum, I explored some of the open tools discussed but none confirmed to the idea of simultaneous drawing capability and though many of them were found to have elaborated features, they appeared to be slow to interact with the server (though they rendered the images faster on the client side, I found that the saving process was anything but slow).

This aim of this tool will be to overcome such limitations making transfer of data between the client and the server faster and synchronized.

The approach is to keep the features very basic that would “'''just solve the purpose'''” of drawings, diagrams, figures, dimensions, free hand sketching and make it shareable among a group (the limit of which can be only worked out as we test, but I guess at least 10) who can simultaneously edit an image.
In fact, I plan to keep the UI so simple that even the six year olds should be able to start drawing the minute they see it, forget about reading any documentation or worry too much about what is placed where.

=== Technology  ===
I propose creating this tool as a java applet using java programming at the server. This is a well thought approach as using java would enable me to write the very basics of pixel rendering methods which can otherwise have various approaches.

Those methods will provide for ways to transmit small sections of updated areas of the shared drawing over TCP, which is to happen very often at a selected time interval or action and is '''“not trivial”''' at all. 

A huge difficulty is making the drawing area shareable in real-time while making sure that the server doesn't get overloaded. This will require careful design of a synchronization mechanism keeping in mind the speed requirements too.

Also, the view is to create a totally re-usable component that can be easily integrated not only with Moodle but any other service that could possibly make good use of it.

A nice example of how many different applets can be collectively used to form a community and collaborate:

[http://code.google.com/p/oekakiapplet/ Oekakia Community]

Here’s how this tool should work:

'''At Server''' - the WYPISWG server program.
The WYPIWYG tool will have a server program that would interact with all the classes for handling different modules of the application such as Shape selection, Color Palette, the display area, text rendering, encoding/decoding etc.

'''At Client''' – WYPISWG applet.
The Applet would display the image inside the browser window of the user and provide a UI for inputs related to various use-cases.

'''The application could be hosted on''' a remote WWW server machine running a HTTP server like APACHE.


==== Why java ====

Performance of an Interactive Web application involves the speed at which the application responds to users, the speed at which client-server interact and the number of users that an application can accommodate.

Let us explore the two major reasons first,

'''Communication:'''
Java scores heavily over AJAX whenever '''major two way communication''' is important. Here, we will require open connections for a long time to facilitate faster frequent transfer of byte streams between server and the client. 

If we are using Ajax then we can give the impression we are continuously connected by querying the server in short intervals, but we are never over a continuous connection, it’s just calling a php/asp/what ever file on the server and getting the response.

Java will always communicate faster once the connection is made than XMLhttpObject (Ajax), '''since using Ajax will always result in a fresh request''' to the web server, including all the overhead.
If set up right, with Java, we can keep a stream open.

Hence I choose Java sockets because the server would need to update all the users connected automatically using the open sockets which require a '''Constant open connection''' (Something similar to the chat-room, where the message when sent to the server, the server updates all open connections with the new message).

'''Customization:'''
Well, nothing beats an applet when it comes to customization. An applet will appear just as you intend it to, to the last pixel.
 
Secondly, ''Cross-browser and cross-platform compatibility'' is one of the main issues in this particular project.

The main aim of any collaborative application remains the ability to log in from anywhere anytime and work. If the user is restricted to any particular browser or operating system, it would simply not serve the purpose of the tool.

Thirdly, java applets can run at a speed that is comparable to (but generally slower than) other compiled languages such as C++, but '''many times faster''' than JavaScript.

I agree that the loading of an applet is found to be slow as compared to the alternatives but that would also depend upon how heavy the applet is. Moreover, after the loading process, it will work better than other approaches for this particular application.
And an applet definitely will be faster, once loaded. :)

The Communication and Customization issues are not resolved with the usage of either flash or HTML5 Canvas element.

HC is really exciting and I already plan to explore Java’s AbstractCanvas too but the current limitations, namely the poor performance on IE, the lack of text drawing, and the bugged gradient support are really annoying for users and developers both.

The limited support and inconsistencies would largely play a spoil sport in an endeavor to make it possible for users to work simultaneously on a task.
For example, there will be serious compatibility issues when multiple users are working on the same image same time on different browsers.

No doubt it is promising but clearly not suitable for this particular task.

With java approach, even those using the lame/old browsers get to use the applet which would significantly make it more popular.

=== Implementation Details ===
I will write a Java Server (a server class plus other classes) along with an Applet which allows continuous connections using Sockets.

The Server should allow multiple connections else only one applet can connect at any one time.
Using synchronized methods (a multithreading concept in java) we can allow only one client applet to pass editing/drawing instructions at a time while other wait in queue.

Drawing will be bitmapped.

There will be different overloaded methods for drawing different shapes beginning from very basic single dot plotting method and then leveraging it to plot rectangles, ovals, straight lines, free-hand drawing, text, etc.

A single public int array (say: matrix) can be used that would hold the pixel information of the entire image at any moment.
The matrix will be updated by various shape drawing methods such as Line, Oval or Dot, etc.

All the different functions to be handled neatly in different classes that would provide clear modularity and ease of code understanding and expansion.
Objects of such classes would simply pass parameters to invoke respective actions.

'''To make byte stream transfers of the edited pixels:'''

*Grab the pixels from the bitmap using PixelGrabber class at repeated intervals.
(PixelGrabber in java can be used to retrieve subset of pixels in an image into a given int array) 
*Create byte stream of RGBs.
*Save it in the main array (matrix) of pixels.

Send drawing instructions when a drawing activity (say a line is drawn) is just over.
Use Synchronized methods to allow processing of only one thread’s edit request at a time.

*Grab the pixels from the sub-region of the image and create a temp array.
*Zip it after creation and send it.
*The file is unzipped after retrieving and pixels are retrieved and copied to another temp array.

*This transferred array of pixels is then plotted over the main array on the other side.
*The similar process occurs at each client and the temp is transferred at a suitable interval (like mouse release, i.e. when the shape is just drawn).

'''Main Objectives while Coding:'''
*To create separate Classes efficiently deploying the OO concepts for modularity and scalability of the code.
*To provide for various parameters in the applet for giving users enough options to keep/use application according to his/her use without worrying about how to modify the source.
*To make faster transfer of data b/w client and server to foster real-time drawing for at least, but not limited to, a small group of users.
*To explore various ways of decoding/encoding and keeping/sending the file over the web.



*I can include various code samples that handle each of the above listed tasks. Kindly comment if they are required at this point.

== '''TimeLine''' ==
I am currently going through my end semester university exams which will end only by April end. I'll get some 4 to 5 days after April 8 (thanks to elections here) in which I would possibly like to provide answers to whatever doubts or implementation related queries that come up.

My summer vacations start by May 2 and I can start my work immediately then.


'''May2 to May 23:'''

1. Research upon already available open applications like [http://www.chibipaint.com/downloads.html Chibipaint] and deciding upon features to both suit the OLPC XO laptops and real-time interaction. To decide upon which one's to lift and which one's to code from scratch and possible integration of codes.

2. Read Moodle Documentation and finalize the overall look and feel of the desired app. Possible integration points at Moodle and/or TinyMCE and the most necessary use cases in coherence with the Integration points chosen.

3. Work out any additional APIs/Technologies to be used. Though I believe I will not have to do any more gathering if I develop it using java.

I have split the entire program time line into ''fortnightly deliverables'' where at the end of each week we can determine the work done, and what and how to proceed to stay on the schedule and get the remaining work completed on time.

'''May 23 to July 6 (Mid term):'''
''F1 to F3 (6 weeks)''

'''F1:''' Implement all the drawing related features individually (so that it is easy to remove/disable features as required) like Shapes, Color pallet, Menus, etc.

'''F2:''' The client side should be totally worked out completely ready with the UI. Depending upon how much I build from scratch this is the final time though I believe it can be easily completed earlier. (Open drawing tools in java will surely help).

'''F3:''' The sending of image/data between client and server. Packing/Compressing/transferring methods (core part of the app). The main goal is to make it fast/synchronized and keep it robust so that there are no inconsistencies.

*By the end of this phase we'll have clear picture of where we are going with the real time drawing.

'''July 7 (Mid term) to Aug 7:'''
''F4, F5 (4 weeks)''

'''F4:''' Deciding upon the number of users, user privileges, optimizing application according to relevance and need of features, Integration and Testing of the application as worked out during community bonding phase. Mainly, all the fitting-adjustment work.

'''F5:''' Encoding/Decoding of the image into the required format. Explore options such as emailing, saving it on the client machine.

'''Remaining time:'''
Polishing code, Removing bugs, Testing, Integration, Documentation, etc.

The current timeline provides enough room for continuous steady development and is mainly kept to correspond to the dates, objectives and guidelines mentioned in the GSoC program. The application with basic functionality can be developed much faster possibly within 4-6 weeks leaving ample time for integration, test and extension.

Although the GSoC program would end by the third week of August, I would like to continue my association with this project and development of Moodle as a whole.

I can also take it up as a project for the entire semester and continue to work on its various dimensions till Nov 2009 if need be.

== '''Goals''' ==
The main aim of any programmer is that his/her work is appreciated and made use of.

I would like to create tools that can further propel Online Teaching and cater more people with varied needs.
I would like to see such a tool realized, re-used, a made better use of.

Additionally, ''exploring java issues'' with OLPC sugar may lead to aid in development of java products for OLPC XO laptops.

''Issues with using other technologies:''

I view this project as a possible mechanism for comparison between traditional java approach and the new upcoming standards like HTML5 Canvas that may lead to a paper discussing pros and cons of each. Hence, without regard to implementation technology I’d like to pursue this project to further enhance my understanding of web technologies and internet browser trends.

== '''Conclusion''' ==

Finally, I hope to provide Moodle with the proposed tool and contribute my bit to it's development.

In future we may see related apps that would totally revamp the chat/A/V integration and put a new kind of virtual class room in place.

I am looking forward to this project as a means to work on a challenging and creative web application that would demand application of my engineering and technological skills.

I am willing to commit myself full-time to this project over the entire summer.

Suggestions for improvement or implementation related queries are most welcome.