Previous Blog Spine Audio Control
Lip Sync - Spine and Papagayo
- animation state
- events
- papagoya lip-sync
- spine plugin support for preview with audio
Demo https://youtu.be/WtlpPBiawa8 Sample Project https://kwiksher.com/daily/blog/lipSync/LipSync.zip
Lip Sync with Papagayo
Papagayo with Spine Export is available from here.
Papagayo 2.0b1 with Spine Export is a fork of LostMoho/Papagayo that adds an exporter for Spine. You need to Qt Creator to build the app. Qt Open Source is here.
Papagayo
- You can choose Spine from Export
- Open JSON button to select your json data of Spine
- Select the bone to attache the mouth
- Import the json data with the mouse and save the project
- MouthShape Slot and Events are there.
- Mouth bone is centered. You need to translate it to correct position.
- Now you can check the mouth animation
Spine plugin support
you can preview animation with audio. You need to start java program to play your audio file and it communicates with Spine timeline.
Start Spine from the command line with --server 4567.
- Windows: Spine.exe --server 4567
- Mac: /Applications/Spine/Spine.app/Contents/MacOS/Spine --server 4567
- Linux: ./Spine.sh --server 4567
Download and run the play.jar file:
- http://esotericsoftware.com/files/play.jar
- java -cp play.jar Play 4567 path/to/music.wav
You will need to have Java installed. When you click Play button on Spine timeline, the animation and aduio are played synclonised.
Export json and atlas
export json and atlas files for Kwik project folder - build4/assets/sprites
Kwik
there are 3 layers * dialog shows the event data that the animation of Spine sends. Each word is dispatched from Papagoya animation. * face will be replaced as Spine animation * bg
Layer replacement > text
dialog layer is replaced as text
Project And Pages > Add Audio
lame.wav from Papagoya tutorial's spoken audio file is added to kwik
- Controls > Loop: times = 1
Project And Pages > add Action
Creates the acitions that will be exected when Spine's animation state sends onStart, onEnd events.
action name is state_start and plays audio lame.wav
action name is state_end and stops the audio playing lame.wav
Custom the file
\build4\components\page01\face_image_.lua to be enabled for importing the spine data.
- create page01 folder under \build4\custom\components\
- copy face_image_.lua to \build4\custom\components\page01
- then edit the three functions. Add extlib.spine library, replace display.newImageRect for spine:newImageRect, add event functions.
local spine = require("extlib.spine").new("face") function _M:localPos(UI) local sceneGroup = UI.scene.view local layer = UI.layer layer.face = spine:newImageRect( _K.imgDir..imagePath, imageWidth, imageHeight) layer.face.imagePath = imagePath layer.face.x = mX layer.face.y = mY layer.face.alpha = oriAlpha layer.face.oldAlpha = oriAlpha layer.face.blendMode = "" layer.face.oriX = layer.face.x layer.face.oriY = layer.face.y layer.face.oriXs = layer.face.xScale layer.face.oriYs = layer.face.yScale layer.face.name = "face" sceneGroup.face = layer.face sceneGroup:insert( layer.face) layer.face.state.onStart = function(entry) UI.animName = entry.animation.name UI.scene:dispatchEvent({name = "action_state_start", entry=entry }) end layer.face.state.onInterrupt = function(entry) UI.animName = entry.animation.name UI.scene:dispatchEvent({name = "action_state_interrupt", entry=entry }) end layer.face.state.onEnd = function (entry) UI.animName = entry.animation.name UI.scene:dispatchEvent({name = "action_state_end", entry=entry }) end layer.face.state.onComplete = function (entry) UI.animName = entry.animation.name UI.scene:dispatchEvent({name = "action_state_complete", entry=entry }) end layer.face.state.onDispose = function (entry) UI.animName = entry.animation.name UI.scene:dispatchEvent({name = "action_state_dispose", entry=entry }) end layer.face.state.onEvent = function (entry, event) UI.animName = entry.animation.name UI.eventName = event.data.name layer.dialog.text = layer.dialog.text .." "..event.data.name UI.scene:dispatchEvent({name = "action_state_event", entry=entry }) end end
#
- layer.dialog.text is updated onEvent function as seen above
- dialog text starts empty and setAnimationByName is added at _M:allListeners func
- animation track 0 and "animation" with looping true
function _M:allListeners(UI) local sceneGroup = UI.scene.view local layer = UI.layer layer.dialog.text = "" layer.face.state:setAnimationByName(0, "Mojo Jojo", false) end
#
dipose is added in _M:toDispose func
function _M:toDispose(UI) local sceneGroup = UI.scene.view local layer = UI.layer layer.face:dispose() end
Publish
press Publish on Kwik Panel to enable the custom file injected to the built codes.