Fall 2007

I marked some of my milestones as complete, not realizing that apparently they disappear from the list when they're complete. If there's a way to get them back, I'd appreciate knowing; if not, please read the next two paragraphs.

 

My biggest accomplishment was rewriting the backend interface. This took a substantial amount of time, but other teams have remarked that it's much easier for them. Incidentally, it's much easier on the backend, too.
I also changed the backend to accept multiple versions of the same piece of content. This same interface is also extensible to accept different formats of content within the same "content" object. For example, once the user uploads a video, you can now store the original data, later on convert it to a different format and save that, save a different resolution of the video, and save a screenshot of it, all under the same "content" object.

Completed video conversion and formatting in the backend using Microsoft Expressions Encoder after a significant amount of research.

Although this was in Kevin's milestone I also researched (even more than video) how to convert images to a specified type (finally settled on using ImageMagick). Kevin is working on another form of creating thumbnails and encoding images that doesn't require external apps, so I won't be committing this yet.

All this is in my branch that I haven't merged into the main branch, and I don't want to mess anything up, so I'll do it during class tomorrow where I can get help.

Along with this new functionality I completely reworked file uploading into an IVisitor that handles content-specific uploading (i.e. videos are uploaded with thumbnails, while text content is not / videos are covnerted using Encoder, images with something else). This allows for extensibility later on in case we need special upload functionality with new content.

I also did a a fairly good job of staying touch with Kevin and Yuan and helping them with issues as they came up.

I had two milestones this week, found here and here.

For the search milestone:

Currently the milestone is stuck, and we are working with the back-end to establish exactly what we want to do. What was originally planned we figured was too complex and slow, so we are working on forming a faster solution. We have implemented a temporary system using enumerations for defining a sort order, i.e. start with name ascending, and on a duplicate, then sort by author descending. This works using a KeyValuePair array, but is just temporary.

Silverlight:
I have upgraded the tag system so it is easy to add tags via javascript in anticipation of sending tags from the backend to the frontend, but this not yet functional since the aspx needs to dynamically insert the javascript. Tags now seek in the video, so you can seek to a tag. Also, the tag timeline can be scrolled separately from the video timeline. Three problems exist with this currently: the buttons are just boxes (I was having trouble with graphics, but this is very easy to fix), there is no scroll bar yet (I have already done similar things so it should be very easy to add), and right now I use a corrective force, so the strength is dependent on the distance, meaning that currently you can only scroll so far. This will (hopefully) be changed to an acceleration system when I get the chance.
I have talked to Sohum about integrating better with the site, and we'll be working on it later. That will not take very long, since it's just a matter of looks.

My milestones are here. I implemented the viewer for markup content. It is in the form of a user control that takes in a string of markup (formatted according to XHTML 1.0 Transitional standards, plus a couple of our own custom tags). Currently it can parse and display the XHTML markups correctly, and also the formatted hyperlink.

1. Finally get something going towards a detailed design of relationships
2. Get RDF/SPARQL implemented
3. Prove that the system will integrate
4. Ensure that people are allocated in the correct way towards the difficult problems
5. Satisfy the customer that we are in a position to meet our goals
6. Work towards being able to have a useable/polished product so that we will be in a position to begin testing on it

Of these, I feel we've accomplished 1, some of 3, some of 4, and some of 6.

1: In particular, we've come a long way toward defining relationships a lot more specifically, including a pretty good sense of how they will be stored and working toward how they will be queried (even if the sparql implementation i'm working from is hellaciously ugly). This took a lot of arm twisting and continual prodding, but it is absolutely postively my highest priority to see this part working ASAP. On the plus side, I think we are now able to add abstract relationships. On the minus side, I think it'll take quite a bit of work to implement a querying system against them using sparql (or some parsed form of it).

2: see 1 commentary mostly. RDF support should not be too hard to add to our data store, although getting a querying engine working against it is my first priority. I feel RDF support is actually rather ancillary as it will only be important for us to be able to provide our relationships in a general form to other sites.

3: In the customer's, and Dr. Wong's, opinion this hasn't really been shown. While it's true that the relationship system is something that really touches many of the pieces of the system as it was originally defined, the existing integration level has given us a system which handles data objects in a uniform way and so save us from many integration issues. In particular, I feel that we have demonstrated a very important instance of integration by allowing a user to upload content, search for that content, and then view that content, which actually takes the same data through all of the original teams' code. Furthermore, our uniform handling of different types of data by the backend spares us of many issues, and our system of identifying and accessing backend objects is already making the relationship model implementation tremendously easier.

4: I believe a number of the criticisms I heard prior to the team rearrangements of the past two weeks were absolutely correct: Content View and Authoring needed to work together more closely, Search had way to much to deal with, and someone needed to deal with unifying the front-end interface. I believe that our new team arrangements will do a lot to help put people where they are needed to work on the difficult problems that haven't been addressed. In particular, Sohum is already doing an exceptional job working through the user interface issues (my Industrial/Organizational Psychology book may say that job enjoyment is only a small factor in performance, but this is certainly not true in a course where the students put in more or less time at their discretion). I have high hopes that putting Corey in charge of Relationships will lead to rapid progress on that front which (sorry Corey), is likely to be in the wrong direction at first, but which will nevertheless teach us things and get us to an answer quickly.

6: Sohum is taking this over at a breathtaking pace, and I hope Luke appreciates the level of improvement we've seen this week (I think this weeks demos will be excellent personally).

 

See 'Additional Comments' for the actual milestones assigned (ie, what this whole P#/G# business is).


G2:I spoke with Aaron a lot, and he got a basic, incomplete but working search from the search tree model done by the Thursday meeting. This worked as a proof of concept, but as the tree is yet again hanging the way of the world), he is instead focusing on SQL and the sorting tree before fully mplememnting the selection tree.

P1/G4:Although the method was "hacked" in the specifics of the code, it did show a proof of concept - and it did work for the customer (if nothing else did). That is, the page existed (though sparse and not exactly 'pretty', but it wasn't supposed to be). The code for getting the aggregated value was in the aspx page instead of added the method to the model, as should have been done, but the neighbors were still retrieved from the model, thus it still showed the functionality. Also, Brad did the code for getting the neighbors, which he really did "hack" and coded sql code in the model rather than where it should have been..But that wasn't my part to be done.
This essentially was allowing the user to form a relationship, and although the code is not there in the backend to add a trust relationship, a rating relationship is fundamentally the same thing so I consider this basically done. Also, Dave is working on the relationship models more, but the actual adding of a relationship is still waiting on the backend (as they add it to a table). However, we were promised a stub-interface by tonight..Haven't talked to Matt since 4 though. So I consider this done, as it is, on our side...but more testing will be required to "put the pieces together" once we get the backend interface.

P2:I feel Derek seemed to understand pretty well what I wanted from him, and by the time he had specific questions, I was back in town. My extra-extensive milestone to him is here.

P3:I talked with Dave in class Friday before i left and went over his milestone with him...I asked him several times, and he seemed to understand his job. He posted his own milestone, which is pretty much what I wanted from him (if in a different order), and can be found .

P4:I made the abstract selection tree extend IVisitorHost and added the necessary Execute method to make it a valid visitor host.

P5/G5:I'm working on this in a branch. The tree form is there, I just haven't fixed the Factory (which should be renamed to be a Parser) yet. I hashed out the design with Aaron 9he came up with most of it). A basic diagram of the new form is the final page (Data Type Selection Tree) of the file Trees.one. For the relationship searching, the "Has" node is used for searching things that are related. For example, a treee of the form Content(And((CreationDate > 9/9/07), Has(AuthorRelationship, Entity(RegisteredDate < 9/8/07))) would return all Content created after 9/9/07 which has an Author-relationship with an enity that was registered before 9/8/07. The notation here might be confusing (with the pareathesis), but it would make sense if I drew it as the tree it is...

My milestone can be found here:Link
Now I have created a plugin for custom tag and enable the user input Guid. The plugin can insert the tag into the content and show a <div></div> section in WYSIWYG editor. When user modify the html version of the content, the tags are shown as <embed />.

- Converted most user controls, (all of those that display content) to use properties to allow parameter passing. The page or control sets the property on a control it contains, and when that control is done having properties set, it updates its values and displays them. This was made easier using abstraction, as it was easy enough to have an AContentControl, that depends on a Content or ContentId before it can display.
- Created the first draft of the markup specification, located here: http://www.bandgap.cs.rice.edu/sites/comp410f07/Private%20Documents/Markup%20Specification.doc
This document specifies the tags and attributes allowed in markup, so we know what to display and the authoring team knows what to allow the user to create. After the authoring team looked at it and not much was changed, it looks like it will work okay. It can always be changed later if necessary.
- Changed the VideoControl to create dynamic Javascript code that passes parameters such as the Guid and tags to the Silverlight player. This required Derek to set up the framework and instruct me on what code to create.
- Set up some basic user controls for displaying and posting comments. These just show static text and have functionality because there is no backend support.

The actual milestone can be found here.
Of the three things that I needed to do, their status is listed below:
1) I was able to come up with a concrete plan for calculating trust, and ratings based on global (aggregate) and local values assigned to each of those. This was written down and e-mailed to Rae. The actual document will be uploaded later today in the appropriate section (although that section will change after today).
2) I wrote out a basic model, sent it to Rae, and wrote out the interface for the RelationshipModel later on, sending it both to Rae and Matt for comments. Later on, I wrote out the stub code for it, and merged it into the main branch. So far there's only stub code, and I haven't changed the actual concrete implementations of the models yet. But this is fair, since actually implementing them would be silly at this point - the interface in the backend is still incomplete, and this much at least needs to be known before more progress can be made. At the very least, Matt knows what is needed, and will be (hopefully soon) forthcoming with the backend interface.
3) I talked with Matt and made sure that everything goes as planned, at least as much as I can from my end. I will talk to him again during class to ensure this once more.

My objectives were:
-Flesh out implementation of author verification to allow multiple authors and store them in the database.
- Talk to Brad and figure out what he wanted for Reviews, Comments, etc. in the database
- Talk with Search team and figure out what they need in terms of RDF /SPARQL support
Of these, the first two are complete. The third is still being worked on, but at least we have an interface we are discussing. Brad and Aaron are also doing some work on this task.
In addition, I was also able to create a Relation table in the database, and add some basic support for accessing it to the backend. Hopefully this will allow us to begin providing some of the "value network" functionality.

The things I did not accomplish are all things which involve close interaction and decisions with other teams. Specifically, they involve how we're going to deal with relationships. Matt, Hubert, Brad, Rae, Derek, and I are actively working on this, and we have--after several aborted attempts--come up with a solution that has potential to work. We don't have time to fully implement this before the deadline for the milestone, but we can continue to work in branched until after the customer meeting.

I haven't merged anything into the main branch or have modified what Sohum moved to the WebApp. That will probably be the first thing I need to do next to make sure that the public skynet3 version works seamlessly.

I also want to clean up tagging because it's very clunky and it doesn't allow any custom tagging.

The search milestone is more deferred rather than uncompleted, but it still needs to be finished.

I need to add the ability to jump between content in the Silverlight tags, but it will be very easy to add once we decide on a design and how I will receive tag data from the back-end, and what exactly exists in tag data. Also, the look has not yet been integrated with the site.

I haven't implemented the embed custom tag, which will allow the user to embed a video content or an image content by setting the attributes type and guid. This may change in the near future and I will implement it accordingly. I also haven't implemented image tagging. I didn't have time to communicate with other teams to finalize what image tag does and how it can link from the image to the target data (i.e. is it going to contain links to other users, other related contents, or both? how is tagging done? what information does an image tag contain?)

2 is still not finished, but I have great confidence that it WILL be working by next week (or perhaps optimistically, Thursday or Friday), which is much slower than I'd hoped, but still acceptable (the sooner we can get this going the better).

 

I think the caveats to 3 and 4 are discussed above.

 

Clearly 5 has been a failure for this week, and I can only hope to do better in the coming weeks.

G1:I'm not certain how it's our job to make sure the backend does something...I did talk to Brad about this, and he said just to ask/pester them about it. I did ask some, but...ya.

G3:This no longer applies ot our group, as review data is not stored in the relationship model. This milestone was assigned before this was realized by Brad.
G5:The searching is not functional yet, as the model was just changed to fit it today, and the backend had other, more pressing matters. Thus it will probably need to be flushed out (a lot), but the basic idea is there. I feel that is all that is actually reasonable for now, as this was orignially assigned for next week (I think).

Search part of the plugin is not finished. The plugin should let user input keyword and search for the content to be added to the editor.
Preview of the content/show content name in the WYSIWYG editor are uncompleted.

- I didn't do the DTD for the markup, since I didn't know if would actually be needed for the XML parsing. It will probably still get done soon though, because it's another nice reference in addition to the specification.
- Showing metadata was low priority, and didn't get done.

Specifically, the item that was left incomplete was 2.
Though the reasoning was kind of explained above, to reiterate, it wasn't fully completed as the interface from the backend is yet unknown. At the same time, I should've spoken with Matt earlier on this and tried to get more done earlier.

The third item still needs work. We need to finalize the interface. Once that is done I'll feel like it was a success, and we can spend later milestones actually implementing the functionality.

An actual implementation of these interfaces remains to be completed, as well as a revision of the ITag interface to hide the tag structure from the frontend.

Detail here is good, but it's a bit unclear whether you completed the video thumbnail and image conversion milestones. Please make it easy for us to compare this report with the team milestone list--as in, painfully obvious.

That is correct. Both are my milestones. I'm in transition between two teams, and one list is a legacy of the old system of giving milestones. -Derek

I appreciate your honesty and detail here, but I really really think it's extremely important that you post your milestones alongside everyone else's. This goes back to advanced planning, and it also contributes to the transparency that Dave discussed in his journal. It applies to you and the team as much as the team and the customer. --Chelsea

This is a great milestone report, and you've obviously made a lot of progress this week. I'm not sure how group milestones ought to factor into your percentage complete, but be sure to discuss that with the other teams to make sure it's uniform across the class. I don't want your grade to be adversely affected by an inconsistency, so I'd consider you to have 100% completion for grading purposes this week.

I couldn't find the WYSIWYG script folder in the project solution this afternoon.

Despite being home this weekend, out and about for most of it, and sick for some as well, I think I got a good bit of work done this week. Unfortunately, not much of what I did has (proper) backend support, so I'm compiling a list of things that are needed from the backend to make my views fully functional. I didn't want to push too much on them this weekend as I was informed by Brad that they had a lot of other groups doing the same, and they took precedence.

A lot of my milestone got delayed because of some ambiguity on the model itself, along with general weekend plans/work. Unfortunately a lot of professors seem to think that the extra time afforded on the weekend can be better spent on...well, their work :P
As an aside to the ambiguity - well, I'm going to have to maintain contact with more people to make sure that sort of thing doesn't happen again.