Wrectangles by David Boudreau

[jchensor]

Hey, guys.

David Boudreau, a long time SF fan (he was prominent during the agsf2 days) wrote a little application that he wanted me to check out and play with. It's an interesting tool that basically lets you create "hit boxes" on an existing video clip. You don't actually create a new video, just a 'script" that can be played alongside the video clip. So if you make a script with all the hit boxes and, in the future, text, as long as someone else has the same video clip you have, they can play your hit box script and see the stuff you made by playing the video with his app.

I told him I'd take a look at it and mess with it, but just haven't had the time (being sidelined by a cold doesn't help). So I figured I would post about it here, since there is no place better. ^_^ Anyone who has some free time can mess with it and see what you think.

======================================

You can get it here:

http://www.lowfierce.com/zrandom/wrecta ... sMar25.rar

======================================

Here's a couple of YouTube tutorial links he made:

http://www.youtube.com/watch?v=SFSuA970_AI
http://www.youtube.com/watch?v=kvfOm9_SqSA

======================================

Here's his original e-mail to me:

* * * * *

Hi James,

I'd like to go ahead and send you the Wrectangles program I've been
working on, in my next email right after I send this to introduce it
first (sending attachment separately in case a spam filter snags on a
.rar file- also, please let me know if you'd prefer a .zip file
instead). I'd really like your feedback, as Wrectangles is one way I
hope I can "raise a glass" to "understand the man" for your efforts
with combo video efforts. You tell me if I'm just "sitting and
staring" off into space though. Here is an INTRO, followed by
INSTRUCTIONS:

INTRO:

Ideally, Wrectangles would be an initial step within the larger scope
of providing more complete diagnostics training that would hopefully
make it into inclusion/integration with a commercial release game
training mode itself, which would also involve the analysis and
reporting of a bunch of detailed statistics and ratios, such as "How
many attacks got stuffed per round, 2.2279". (possibly customize
stats into custom ratios reporting... user community encouraged to use
it to find most meaningful statistics, eg which stats tend to be
highest when I win? what is consistent about the best players' stats
to extrapolate trends? etc.)

Diagnostics training would also include the actual hit/vulnerable
locations (not our estimates, which so far we can only estimate with
tools like this one) on frames during matches, as well as an onscreen
display of inputs (i.e. anytime you hold the stick left, that is
indicated in a small area at bottom corner of screen and when you stop
holding left, you can also see that clearly), which may help when
practicing combos (e.g. "why didn't it link that time? oh, because the
button press didn't register there, maybe I tend to be too early on
it")

But that's all still pie-in-the-sky. Here's what Wrectangles is so far:

High Level:

-Now, we can already watch videos of matches, study them to review
errors and misses to improve, or to demonstrate combos, etc.

-Wrectangles intended as a simple tool to provide more capabilites and
provide more options for that process; more granularity... a kind of
"interactive machinima-slash-demotool"

-Combos, setups, precise distancing, zoning/controlling space, and
strategy can be explained, demonstrated, tutorialized and shared in
more detail with Wrectangles.

Workflow (see Instructions below for details before trying):

File-New --> Select source movie file --> Grab some frames
--> Edit (add rects/text) ... save, share file/discuss

File-Open --> Select .wre compilation file --> Review/Re-edit

Benefits:

-Shuttle-jog through clips at various speeds
-Illustrate *and*animate* hit/vulnerable areas of moves
-Explain/make tutorials of combos/setups/shutdowns/zoning /corner traps
-Recognize/Convey subtle differences in distancing more easily and
clearly (than via text alone on forums, for example)

Features:

-Grab short clips from your source videos of rounds (.wmv,.avi,.mpg
DirectShow-able files in 320x240 or 640x480)
-Shuttle Jog capabilities on those frames (control playback speed to
slow down framerate, pause exactly on frames you want to review, and
frame-by-frame advance/rewind)
-Draw rectangles over a move's animation to indicate precise
hit/vulnerable areas for each character, on each frame
-Add text comments, specific to any particular frame
-Note the name/handle of player1, player2
-Save to tiny .wre file to share and discuss analysis with others
that have the same source videos

Limits:
First release, still a little rough around edges: Windows only. Can
only grab max of 1000 frames (about 30 seconds' worth) and that may be
further limited by your cpu/gpu/vram; cannot save compilation file out
to a second avi file yet, just an ASCII file requiring source movie.
Users can then share the ASCII file and other users reprocess clip
from same source movie. Any sound in source won't be in compilation
file. Does not do anything special with codecs or compression;
processes frames in VRAM (your graphics card). If you have the latest
high-end machines, note the 30 seconds limit because it was developed
on an old machine, so this may be lifted.

Cost:
Free. Please share copies with others soon. Ask if interested in source code.

**********************************
INSTRUCTIONS:

Unarchive the attached file (in next email) WrectanglesMar25 and note
two executables and two folders. The Wrect640x480.exe can do both
movie file sizes 320x240 as well as 640x480, so use that generally,
but the 320x240 exe gives a bigger screen when maximized for movie
files in that smaller size). Put some movie file(s) you have (of 320x
or 640x size) in the "SourceMovies" folder, and start the
Wrect640x480.exe. The source movies have to be in the SourceMovies
folder under the exe.

Press space to begin, go File-New and select one of your source
movies. It should begin to play. Hold down the "G" key on your
keyboard for a few seconds (up to about 5 secs' worth is plenty the
first time you try it, later you can go up to 30 seconds, or 1000
frames). As soon as you let go of the G key, it will process by
playing the movie again from the beginning and grab the frames. Let
it play all the way through "Processing..." until you see the
red/blue/green/etc. boxes at the bottom of the screen.

Now you can use P key to toggle pause/play, arrow keys to speed
Up/Down, and while paused, Left/Right to rewind/advance
frame-by-frame.

While PAUSED on a frame, you can draw a rectangle with the mouse (e.g.
draw over an arm punching) in a selected color from the bottom (just
click on one of the colors to choose). You should see the rectangle's
green handles for positioning it (top left) and resizing (bottom
right). To bring a rect into focus, click in its positioning handle.
While you see the handles of a rect, you can extend it to the
subsequent frame by holding Shift key and press right arrow. Start
drawing rects on the first frame of an attack, and then extending
(Shift+rightArrowKey) to the last frame. If you go too many frames,
go to the highlighted rect's last frame it appears on and hold Shift,
and press left arrow key to reduce a frame that that rect appears on.
To actually delete a
rect, highlight it's handle and press Del key. The process isn't
perfect as sometimes the characters move or get pushed but rects stay
in the same spot throughout all frames, so for those times, a
workaround is: multiple rects can be drawn on different frames, so
when the character starts scrolling/gets pushed back, don't
Shift-Right to the next frame, but Right arrow key only and draw a new
rect (the rects can exist on multiple, sequential frames, but each has
only one position). Remember, this is all while PAUSED.

Speed of playback: P key toggles pause/play; paused is equivalent to
a "Drag" parameter of 700, and normal play speed where Drag=30.
Up/Down arrow keys to get in between. (I call it Drag param because
it's the number of milliseconds it waits before proceeding to the next
frame).

You can go File-SaveAs to save (no Save this release). What you are
saving is a .wre file, a "compilation" of the designated source movie
frames and the rects you drew on it. For now don't open any of the
ones you see in the archive I send in the Compilations folder, cause I
didn't include those source movies (to save space in the archive).
Whenever a .wre is openned, it will reprocess from the same source
movie which needs to be in your SourceMovies folder.

Unless it's processing something, you can go File Open to open a .wre
compilation file. It will first process from source movie
automatically and then you can play it back to see it with the rects
on it-- but *animated*. That's basically what it's all about, hope
that's not too complicated.

Next basic feature not yet implemented: the ability to add comments
(text) on any given frame.

P- pause/play

ARROW keys- control speed/frame-by-frame (or -/+ with F/R keys: F/R
keys also does the same as left/right keys, and similarly, numpad's
minus/plus keys same as down/up.)

E- close a source movie (without grabbing anything)

G- grab frames after openning a source movie (file-new)

--David
(XSPR/mrdhalsim)

btw: you can take a look at the .wre file structure in Notepad if
you're curious, but I don't want to mention this because users may
mess something up as you can imagine... as it's not 100% forgiving in
the code but maybe someday it may be... you can see a faint
resemblance to xml structure which may or may not be useful if
interest in this grows....

The last tournament I went to last week was actually a
(pre?)qualifier connected to Tougeki/SBO for AE (2 on 2). I didn't
have a partner, but happened to meet my old friend/rival from A3 days,
443 and recruited him. We got eliminated early by the guys that won
it (single elim), they drove 3 hours for it and one plays with Muteki
Guile. I hadn't practiced much at all, instead working on Wrectangles
since last tournament, and after this tourny finished, gave a demo of
Wrectangles on my laptop and handed out 10 CDs, telling everyone to
share/copy but
unfortuantely my compiler can't handle Japanese/UTF characters and
everything's in English... so not sure on the reception it'll get
here.

btw I can send you/upload somewhere a demo video, of me using
Wrectangles and voiceover of what I'm doing, but the file would be 20
to 30MB.

-David

[Maj]

Who gets the most out of something like this? Obviously it wouldn't help with making combo videos because those hitbox measurements are imprecise at best. Maybe if someone was incredibly bored, they could go through their favorite game and create sort of a database of approximate hitboxes and footnotes about each move. Even that seems difficult to navigate, if intended as an index/glossary.

It also wouldn't really help make the videos better because most of us can find ways to include those hitboxes in the editing stage. That would cut down the amount of stuff that the audience has to download and install in order to view the video properly.

So it seems like this is intended for people who don't have video editing experience or capability, who have videos already, and who want to annotate them for whatever reason. Those CvS1 CaliPower vs jchensor match videos posted on SRK way long ago come to mind, because many of the accompanying match notes could be conveyed better with this program. But that's kind of a crowd that we don't have, right? How many people actually volunteer to take apart matches step by step? How many people are even knowledgeable enough to do something like that effectively? I mean, it involves kind of a lot of guessing about what's going on in the player's minds, and it takes a lot of underdstanding for those guesses to be accurate.

The most useful application for us that i can think of, is the use of Wrectangles as a way of making "Director's Commentary" type notes for a combo video release. Many of us already write up lengthy transcripts or guides explaining our videos, and this might be a more interesting way of conveying all of that information. Still, it seems like a lot of work and i wonder how many people would actually go through the trouble of installing this to see it. Even without the trouble of having to install a new program, i wonder if enough people are even interested in rewatching a video multiple times just to hear our commentary on it. Cuz it would probably take quite a while for us to make a decent commentary track.

I dunno ... what do you guys think?

[fullmetalross]

I generally agree with yo Maj. While Approximate hitboxes are useful for demonstrating certain moves we know as having distended hitboxes such as the tip of kyo's far rh and such. Otherwise whats the point of drawing the hitbox on the sweep if you don't know where it is? Also how accurate is the frame capture in this? If you can't get all the frames of animation in there then how can we count frames to make sure we are turning hitboxes "ON" and "OFF" at the correct times?

It seems sort of useful at first but then on further inspection I think we get alot less out of it.

[ZenFire]

Iread this when you first posted it and I had the exact same thoughts as Maj but I didn't post anything cus I still believe "if you can't say anything nice...." you know the rest.

[Maj]

Well, the commentary track idea isn't too shabby. I'm just wondering if it's worth the effort, considering who it would benefit. Cuz really, most of the people i'm trying to impress with my videos don't really need an annotated version to figure out what's going on.

So kinda by definition, that means the commentary would be for people who aren't deeply invested in the game(s) i'm using. Then the question becomes, would those people bother going through all this trouble (or rather any minor amount of trouble) to get the benefit of all the effort this would require on my part?

[ZenFire]

maybe he's planning on making it exportable to AVIso you don't ned t he program, that wouldmake it more accessible. But like you said, there are other ways of doing that, and they're more flexible.

[Maj]

So then what's the intended audience for this program? I guess if someone is trying to make everyone recognize a new game, they could use this program to point out interesting things that happen during the match. That would make a much more effective advertisement than simply making people watch match vids or combo vids, because it's really easy to miss whatever it is that makes the game special. This program would make it much easier to point out the various options of the player and also the limits being overcome in the video.

I could imagine this being a useful tool for supporters of underrated games like GGX series, Virtua Fighter, or doujin games.

[Xenozip.]

Discovering the details about hit-boxes and hittable-boxes has always interested me in regards to both combo and tutorial video creation. Many many times I've wished I could see the exact areas for both hit and hittable boxes for moves in games. Main reason for wanting this is so that you don't have to do countless amounts of testing on certain theoretical combos just to see if something is even possible or not, which would be really useful for creating exotic combos. Although I could see something like this also being really useful for tutorial videos so that you could literally show a visual representation of how and why moves do what they do.

This tool seems more useful for creating tutorials rather than combos, IMO. It only displays what you've already run across, it doesn't display any "hidden knowledge", so to speak.

On the other hand, if a tool was able to extract the exact data for a hitbox, that would be incredibly useful. I wonder how the hitbox data in Mooks like the "All About Street Fighter Zero 3" are created. I assume it comes from the source, but I still wonder how. :B

If I had been able to visually see the hitboxes and hittable boxes in Melty Blood, for example, it would not have taken very long to figure out some really exotic "backwards" combos, such as the 5/2A infinites. As it is (or was?) in Melty Blood, if you cancel one attack into another your character will remain facing the same direction, even if the opponent crossed up. Oddly, the pushback caused from a connected hit is also relative to which direction the attacker is facing, not the person being hit. Furthermore, some attacks had large enough hitboxes that they actually occur behind the player as well as in front (like 3S Yun's toward+Fierce). Combine the two and you had the potential to do 2/5A infinites by forcing a crossup and repeatedly canceling your A attack. The A attack hits the player behind the attacker so the player behind hit moves toward the attacker instead of away, yet the attacker doesn't turn around.

However, that was not the only thing possible. Melty Blood actually had a lot of really interesting hitboxes that hit from both sides and allowed some really exotic (sometimes impractical, sometimes practical) "backwards" combos.

There's also situations in other games where an extremely detailed knowledge of hitboxes can lead to interesting link combos, such as 3rd Strike. I believe in KYSG's Elena combo video he utilized Elena's slide against Elena's back+Roundhouse move, the slide move crossed under the roundhouse move, but the roundhouse still connected with Elena's hittable box and lead to a cross-up link combo, where normally it would not be possible to follow up back+Roundhouse.

Even in games like SFA3, a visual representation of hitboxes and hittable boxes would be very nice to have for many different reasons. However SFA3 has been out for so many years and has been play-tested in a variety of ways by a variety of players, so there probably isn't much left to discover in that game. But for a game like CFJ it might be possible to discover some really interesting properties for moves if there was a visual representation for all the characters attacks.

But I digress. I'll stop aimlessly rambling now.

[jchensor]

I e-mailed him recently and asked him about the potential for adding freeze frames and slow down functionality in the "playback." I can't see this tool being greatly useful only because the hit boxes would be on screen for such an incredibly short period of time.

But if he adds Text functionality, as well as freeze frames (so a script you execute on top of an existing movie will allow you to freeze frame the movie while still animating rectangles or text) could be really useful for analyzing match videos or allowing you to make director's commentaries or explanation scripts without having to render more than one video.

Unfortunately, without freeze frame and slowdown, I'm not sure how useful this tool can be.